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TRAINING  CAPABILITY  DATA  FOR  DISMOUNTED  SOLDIER  TRAINING  SYSTEM 
EXECUTIVE  SUMMARY 


Research  Requirement: 

The  U.S.  Army  recently  fielded  a  Dismounted  Infantry  simulator  to  train  small-unit 
tactical  skills  and  to  link  small-unit  simulation  training  with  combined  arms  simulation.  The 
Dismounted  Soldier  Training  System  (DSTS)  is  intended  to  enhance  training,  replicate 
battlefield  conditions,  balance  resources,  and  sustain  readiness.  In  order  to  determine  the  extent 
to  which  Dismounted  Infantry  simulation  in  general  and  DSTS  in  specific  provides  realistic  and 
useful  training,  the  Maneuver  Center  of  Excellence  conducted  two  capabilities  experiments  in 
the  summer  and  fall  of  2012.  The  U.S.  Army  Research  Institute  was  asked  to  collect  and  analyze 
training  capabilities  data  from  the  experiments.  The  purpose  of  the  data  analysis  was  to  describe 
the  training  capabilities  of  immersive  Dismounted  Soldier  simulation  (i.e.,  DSTS).  This 
description  of  capabilities  augments  and  extends  previous  summaries  of  Dismounted  Soldier 
simulation  capabilities.  The  present  report  was  intended  as  descriptive,  and  comparisons  were 
not  made  among  the  capabilities  data  or  among  DSTS  and  other  simulation  systems. 

Procedure: 

The  two  multi-day  capabilities  experiments  provided  the  opportunity  to  obtain  Soldier 
input  on  the  ability  to  move,  shoot,  and  communicate  in  DSTS  and  to  utilize  DSTS  for  specific 
training  outcomes.  The  first  experiment  was  the  User  Assessment  of  DSTS  prior  to  fielding. 

The  second  experiment  assessed  training  capabilities  of  DSTS  as  part  of  Joint  Eorces  Bold  Quest 
2012.  In  both  experiments.  Soldiers  received  familiarization  training  for  DSTS,  conducted  a 
squad-level  baseline  mission  in  DSTS,  conducted  squad-level  training  for  the  mission,  conducted 
an  evaluation  mission,  and  finally  completed  a  series  of  questionnaires  about  their  experiences 
using  DSTS.  The  primary  data-collection  instrument  was  a  DSTS  performance  capabilities 
checklist,  which  assessed  Soldiers’  perceptions  of  their  abilities  to  execute  individual  tasks  and 
collective  tasks  in  the  simulation  as  compared  to  real  life.  In  addition  to  the  capabilities 
checklist,  other  instruments  captured  Soldiers’  perceptions  of  the  impact  DSTS  training  had  on 
training  outcomes,  decision  making,  preparation  for  mission  execution,  and  operational  realism. 
Bold  Quest  2012  also  provided  a  comparison  of  DSTS  training  to  live  training. 

Bindings: 

The  results  could  be  classified  as  “mixed”  at  best.  Performance  capabilities  were  mostly 
classified  as  acceptable  in  the  User  Assessment,  but  the  majority  of  performance  capabilities 
were  rated  as  unacceptable  in  Bold  Quest.  Likewise,  Soldiers  in  the  User  Assessment  reported 
feeling  immersed  in  DSTS,  but  Soldiers  in  Bold  Quest  reported  a  lower  sense  of  operational 
realism  in  DSTS  compared  to  live  training.  Not  only  was  the  sense  of  operational  realism  lower 
in  DSTS  than  live  training,  but  also  the  sense  of  training  preparation  was  lower  in  DSTS  than 
live  training  during  Bold  Quest.  The  discrepancy  in  Soldiers’  perspectives  of  DSTS  across  the 
two  experiments  emphasized  the  high  level  of  system  familiarity  required  to  obtain  training 
benefit  from  DSTS.  More  than  8  hours  of  structured  familiarization  training  were  required 
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before  Soldiers  were  eomfortable  enough  with  DSTS  to  effeetively  exeeute  training.  However, 
the  DSTS  AAR  eapabilities  were  universally  pereeived  as  providing  a  positive  impaet  on 
training. 

Utilization  and  Dissemination  of  Findings: 

The  data  doeumented  in  this  report  were  used  to  inform  deeisions  from  the  DSTS  User 
Assessment.  Some  of  the  data  were  also  ineluded  in  the  Army  Expeditionary  Warrior 
Experiment  -  Bold  Quest  2012  report  (U.S.  Army  Evaluation  Center,  2013).  In  addition,  the 
results  were  briefed  to  Program  Manager  -  Combined  Arms  Tactieal  Trainer,  Training  and 
Doetrine  Command  Capabilities  Manager  -  Virtual  Training,  and  Maneuver  Center  of 
Exeellenee  Direetor  of  Training  and  Doetrine. 
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TRAINING  CAPABILITY  DATA  FOR  DISMOUNTED  SOLDIER  TRAINING  SYSTEM 


Introduction 


Background 

One  of  the  U.  S.  Army’s  top  seienee  and  teehnology  investment  areas  and  strategies  for 
the  2020-2030  timeframe  is  to  give  fire  and  maneuver  forces  a  combat-ready  edge  through 
individual  and  collective  training  with  simulations  and  devices  while  in  garrison  and  during  non¬ 
standard  deployments  (Department  of  the  Army,  2012).  To  support  this  investment,  the  U.S. 
Army  recently  fielded  a  Dismounted  Infantry  simulator  to  train  small-unit  tactical  skills  and  to 
link  small-unit  simulation  training  with  combined  arms  simulation.  The  Dismounted  Soldier 
Training  System  (DSTS)  uses  a  helmet-mounted  display  and  controllers  mounted  to  the 
surrogate  weapon  to  allow  Soldiers  to  interact  with  the  virtual  environment.  The  use  of  DSTS  is 
intended  to  enhance  training,  replicate  battlefield  conditions,  balance  resources,  and  sustain 
readiness. 

Virtual  simulation  could  provide  a  useful  tool  to  enhance  the  execution  of  individual  and 
collective  tasks  because  of  the  capability  to  recreate  situations  and  environments  that  cannot  be 
replicated  in  live  training  and  the  ability  to  rapidly  modify  training  conditions.  DSTS  will  also 
allow  ground  troops  to  participate  in  a  common  virtual  environment  with  armor  and  aviation. 

The  Infantry  Soldier  focuses  on  basic  tasks  when  in  his  operational  environment.  He  moves,  he 
shoots,  and  he  communicates.  At  a  minimum,  simulation  training  must  exercise  these  skills  at 
both  the  individual  and  collective  level. 

In  order  to  determine  the  extent  to  which  Dismounted  Infantry  simulation  in  general  and 
DSTS  in  specific  provides  realistic  and  useful  training,  the  Maneuver  Center  of  Excellence 
conducted  two  capabilities  experiments  in  the  summer  and  fall  of  2012.  The  first  experiment 
was  the  User  Assessment  of  DSTS  prior  to  fielding.  The  second  experiment  assessed  training 
capabilities  of  DSTS  as  part  of  Joint  Forces  Bold  Quest  2012.  The  U.S.  Army  Research  Institute 
was  asked  to  collect  and  analyze  training  capabilities  data  from  the  experiments.  The  purpose  of 
the  data  analysis  was  to  describe  the  training  capabilities  of  immersive  Dismounted  Soldier 
simulation  (i.e.,  DSTS).  This  description  of  capabilities  augments  and  extends  previous 
summaries  of  Dismounted  Soldier  simulation  capabilities  (i.e.,  Campbell,  Knerr,  &  Lampton, 
2004  ;  Knerr,  2007;  and  Lampton  &  Jerome,  2010).  Because  this  report  was  intended  as 
descriptive,  comparisons  will  not  be  made  between  the  capabilities  data  and  training 
performance  or  among  DSTS  and  other  simulation  systems  (e.g.,  DSTS  and  Small  Unit  Virtual 
Immersion  System).  Where  possible,  some  interpretations  will  be  made  about  the  data,  but  these 
interpretations  are  intended  to  provide  better  context  for  understanding  the  data  rather  than 
inferences  about  the  importance  of  the  data. 

Dismounted  Soldier  Training  System 

DSTS  Development.  DSTS  evolved  from  a  previous  Army  Science  and  Technology 
Objective  (STO)  to  develop  virtual  environments  for  dismounted  Soldiers.  The  STO,  titled 
“Virtual  Environments  for  Dismounted  Soldier  Simulation,  Training,  and  Mission  Rehearsal,” 
focused  on  “overcoming  critical  technological  challenges  that  currently  prevent  high  fidelity 
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dismounted  soldier  simulation”  (Knerr,  et  al.,  2003).  Yearly  objectives  were  to  integrate  and 
evaluate  the  technologies  developed  during  the  year.  Multiple  reports  were  published  that 
captured  the  improvements  to  simulators,  the  virtual  environment,  and  the  development  of  an 
after  action  review  (AAR)  system  (e.g.,  Knerr,  et  al,  2002;  Pleban  &  Beal,  2002;  Pleban,  Eakin, 
&  Salter  2000;  Pleban,  Eakin,  Salter,  &  Matthews  2001). 

Several  types  of  simulators  and  technologies  were  assessed  over  the  course  of  the  STO 
including  man-wearable  ensembles,  desktop  systems,  an  omni-directional  treadmill,  and 
surrounding  projection  screens.  Eventually,  one  system,  the  prototype  Soldier  Visualization 
Station  (SVS),  was  identified  as  the  default  system  of  choice  to  enable  the  Soldier  to  shoot, 
move,  and  communicate  in  virtual  environments.  This  system  became  the  focus  of  the  next  four 
years  of  assessment  and  development.  Systems  continued  to  evolve  through  assessments  and 
subsequent  recommendations  of  features  that  should  be  incorporated  into  the  next  generation  of 
systems.  The  result  was  a  system  based  on  a  networked  series  of  immersive  and  desktop 
simulators  coupled  with  an  AAR  system,  an  enhanced  dynamic  terrain  server,  and  an  improved 
semi-automated  force,  that  enabled  an  Infantry  Squad  to  execute  squad  collective  tasks  (see 
Knerr,  2007,  for  a  review  of  the  STO  development  and  assessment  results). 

After  the  STO,  additional  technological  improvements  bridged  the  gap  between  the  SVS 
and  the  production  of  DSTS.  The  most  important  improvement  was  the  man-wearable  computer 
that  allowed  Soldiers  unrestricted  (or  less  restricted)  movement  in  physical  space  in  order  to 
better  interact  with  and  in  the  virtual  space.  The  final  iteration  of  prototype  systems  was  released 
in  2007  and  called  the  Virtual  Squad  Training  System  (VSTS).  Eike  some  of  its  predecessors, 
VSTS  included  a  combination  of  man-wearable,  tethered,  and  desktop  interfaces,  workstations 
for  the  BattleMaster  and  Semi- Automated  Eorces,  and  AAR  capabilities.  The  man-wearable 
systems  demonstrated  advantages  in  individual-Soldier  movement  and  in  weapons  use. 

However,  the  man-wearable  systems  showed  only  minimal  training  effect,  and  Soldiers  still 
reported  difficulty  with  spatial  hearing  and  interaction  with  objects  in  the  virtual  environment 
(Knerr,  Garrity,  &  Eampton,  2004;  Eampton  &  Jerome,  2010).  A  snapshot  of  the  evolution  of 
DSTS  can  be  seen  in  Eigure  1 .  The  timeline  does  not  include  all  of  the  developed  dismounted- 
infantry  simulation  technologies,  but  it  does  highlight  some  of  the  major  efforts  and  lists  the 
simulators  whose  characteristics  contributed  to  the  current  DSTS. 

DSTS  Characteristics.  DSTS  suites  consist  of  nine  man-wearable  Virtual  Soldier 
Manned  Modules  (VSMM),  five  desktop  Virtual  Soldier  Multi-Eunctional  Work  Stations 
(VSMW),  a  Semi- Automated  Eorce  (SAE)  workstation,  an  Exercise  Control  (EXCON) 
workstation,  and  an  AAR  station.  DSTS  surrogate  weapons  mix  includes:  5  x  M4  Rifles,  2  x 
M4/M320  Rifle/Grenade  Eaunchers,  and  2  x  M249  Machine  Guns.  The  DSTS  virtual 
environment  is  generated  in  Virtual  Battle  Space  2  (VBS2)  and  was  initially  fielded  with  three 
terrain  databases.  Multiple  DSTS  suites  can  be  networked  together  for  Infantry  Platoon 
operations  and  eventually  will  have  the  capability  of  networking  to  the  Close  Combat  Tactical 
Trainer.  The  system  is  designed  to  be  portable. 
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Experiments  and 
Assessments 

Evaluation  of  the  Virtual  Squad  Training  System  | 

An  Assessment  of  V-IMTS  (2004)^/'  V-IMTS 
An  Assessment  of  Wearable  Virtual  Simulators  (2004)^^VS2-DI,  DAGGERS,  ASWETS 

Joint  STO-  Virtual  Environments  for  Dismounted  Soldier 
Simulation,  Training,  and  Mission  Rehearsal  (1999-2002)  ‘ 

DWN-ERT  (Enhanced  Restricted  Terrain)  (1998)/'soidier  Visualization  Station 


Dismounted  Warrior  Network  (DWN)  Project  (1997^ 

TRAC-WSMR  Soidier  Station 
Bayonet  w/Omni  Directional  Treadmill 

Perceived  Usefulness  of  TTES: 

A  Second  Look  (19951^'  USMC  Team  Target  Engagement  Simulator 
Dismounted  Soldier  System 


DS  Simulators 


SIMNET 

1980^  ''dARPA  Tank  Team  Gunnery  Trainer  (TTGT) 
DARPA  Tank  Gunnery  Trainer  (TGT) 


Systems  based  simulators 


Figure  1.  A  Timeline  for  Dismounted  Soldier  Simulator  Development  and  Assessment.  ^ 

Virtual  Soldier  Manned  Module.  The  VSMM  is  the  immersive  interface  for  DSTS  (see 
Figure  2).  The  VSMM  operates  on  a  wireless  network  allowing  Soldiers  and  weapons  to  be  un¬ 
tethered  from  any  external  equipment.  Each  Soldier  wears  a  backpack  computer  that  generates 
the  virtual  environment  (VE)  and  a  helmet-mounted  display  (HMD)  to  view  the  VE.  The  HMD 
has  five  different  controls  and  adjustments  to  assist  in  fitting  it  as  close  to  the  eyes  as  possible 
without  resting  on  the  nose.  It  provides  the  Soldier  with  a  360  degree  horizontal  field  of  regard, 
a  180  degree  total  vertical  field  of  regard,  a  60  degree  instantaneous  horizontal  field  of  view 
(EOV),  and  a  45  degree  instantaneous  vertical  EOV.  Eull  peripheral  vision  is  not  achieved.  The 
Soldier  stands  on  a  4-foot  diameter  ergonomic  pad  with  virtual  movement  controlled  by  a 
thumbstick  located  on  the  vertical  handgrip  of  the  surrogate  weapon.  Body  sensors  attached  to 
the  Soldier  (e.g.,  three  sensors  per  arm  and  one  sensor  per  leg)  translate  physical  movement  of 
arms  and  head  into  virtual  movement  of  arms  and  head.  Soldiers  communicate  through  a  headset 


*  ASWETS  -  Advanced  Soldier  Wearable  Embedded  Training  System,  CCTT  -  Close  Combat  Tactical  Trainer 
DAGGERS  -  Distributed  Advanced  Graphics  Generator  and  Embedded  Rehearsal  System,  DSTS  -  Dismounted 
Soldier  Training  System,  SVS  -  Soldier  Visualization  Station,  SVS2-DI  -  SVS2-Dismounted  Infantry,  TRAC- 
WSMR  -  TRADOC  (Training  and  Doctrine  Command)  Requirements  Analysis  Center- White  Sands  Missile  Range, 
V-IMTS  -  Virtual  Integrated  MOOT  (Military  Operation  in  Urban  Terrain)  Training  System,  VIRTSIM  -  Virtual 
Simulation,  VSTS  -  Virtual  Squad  Training  System 
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and  microphone.  The  VSMM  utilizes  radio  frequency  identification  (RFID)  tags  and  hand 
sensors  to  allow  the  Soldier  to  select  and  use/throw  additional  items  such  as  a  global  positioning 
system  (GPS)  and  grenades. 


Figure  2.  DSTS  Virtual  Soldier  Manned  Module  Ensemble. 

Desktop  workstations.  The  VSMM  is  supported  by  four  types  of  networked  desktop 
VBS2  terminals.  First,  the  VSMW  is  used  to  replicate  combat  multipliers  in  support  of  the 
Infantry  Squad.  Operators  of  the  VSMW  can  fill  the  roles  of  machine  gun  teams,  vehicle  crews, 
or  other  roles  within  higher  echelons.  Second,  the  SAP  workstation  has  the  capability  of 
controlling  single  or  multiple  SAF.  Operators  of  the  SAF  workstation  have  a  dual-purpose 
function  as  a  SAF  controller  or,  if  necessary,  another  VSMW  workstation  operator.  Third,  the 
EXCON  workstation  controls  both  the  system  and  the  training  scenarios.  The  EXCON 
workstation  operator  controls  the  DSTS  hardware  to  include  powering-up,  initialization, 
troubleshooting,  and  monitoring  of  the  system  over  the  network.  The  operator  is  also  responsible 
for  modifying,  loading,  and  running  each  scenario  in  support  of  the  training.  Finally,  the  AAR 
Station  uses  the  VBS2  AAR  capability  to  record  and  playback  the  scenario.  The  station  consists 
of  two  large  flat  panel  displays,  a  keyboard,  and  a  mouse.  Detailed  information  concerning  the 
VBS2  AAR  capabilities  is  documented  elsewhere  (i.e.,  Topolski,  Green,  Leibrecht,  &  Rossi, 
2011)  and  not  discussed  here.  An  example  layout  of  the  complete  DSTS  is  provided  in  Figure  3. 
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Virtual  Soldier  Multi-Functional 
Workstations  and  Semi-Automated 

Force  Workstation  Exercise  Control 


Figure  3.  Example  Dismounted  Soldier  Training  System  Layout. 

Body  position  and  movement.  Soldiers  are  matched  to  pre-determined  generic  avatars 
prior  to  initial  calibration.  The  sensor  harness  combined  with  calibration  enables  the  Soldier  to 
enter  and  control  his  avatar  within  the  VE.  The  HMD  sensor  captures  and  replicates  the 
direction  the  Soldier  is  looking  in  the  real  world,  while  the  arm  sensors  capture  and  replicate  the 
Soldier’s  arm  movement  in  the  real  world.  Soldiers’  physical  movements  from  standing  to 
kneeling  to  prone  are  captured  by  leg  sensors  and  mirrored  by  their  avatar  in  the  VE. 

Additional  actions  such  as  mounting  a  ladder  and  employing  a  Claymore  mine  are  controlled  by 
four  buttons  located  on  the  vertical  handgrip  of  the  surrogate  weapon  that  control  an  action  menu 
that  appears  in  the  HMD. 

Rate  of  movement  is  dictated  by  the  amount  of  pressure  imposed  on  the  thumbstick  in 
combination  with  the  position  of  the  weapon  (i.e.,  high  ready  or  low  ready).  The  Soldier  is  able 
to  move  in  any  direction  within  the  VE  by  physically  turning  the  body  towards  the  desired 
direction  of  travel.  Low  crawling  or  crouching  and  moving  is  accomplished  by  activating  the 
thumbstick  while  physically  in  the  prone  or  kneeling  position. 

Ereedom  of  movement  within  the  VE  is  only  limited  by  the  size  of  the  terrain  database 
and  the  objects  positioned  within  the  database  (e.g.,  walls,  barriers,  trees,  etc.).  Objects  placed 
within  the  terrain  database  hamper  movement  in  the  VE  similarly  to  obstacles  in  the  real  world. 
That  is,  the  avatar  cannot  walk  through  walls,  trees,  or  any  other  immoveable  object  in  the  VE. 
However,  the  Soldier  could  step  over  objects  that  were  low  enough  (e.g.,  window  sills,  low 
walls,  etc.).  Movement  within  an  urban  environment  is  restricted  by  the  proximity  of  the  avatar 
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to  the  building  walls.  The  laek  of  a  tactile  indicator  causes  the  movement  of  the  avatar  to 
become  erratic  when  trying  to  stack  against  a  wall  to  enter  and  clear  a  room,  and  avatars  can 
become  stuck  in  the  wall  and  have  to  extricate  themselves  before  continuing  to  move.  Soldiers 
are  capable  of  opening  and  closing  doors  with  either  a  hand  movement  (i.e.,  physically  reaching 
towards  the  door  and  twisting  the  wrist)  or  an  action  menu.  Once  inside  a  building,  Soldiers  can 
move  up  and  down  stairs  and  in  and  out  of  rooms. 

Navigational  aids  are  available  when  programmed  into  the  RFID  tags.  Soldiers  can  be 
issued  a  lensatic  compass  or  GPS  based  on  leadership  discretion.  These  items  could  be  activated 
during  the  scenarios  by  placing  either  hand  over  the  RFID  tag;  the  hand  sensor  would  detect  the 
RFID  signature  and  display  the  item  in  the  HMD.  The  lensatic  compass  floating  dial  moved  to 
the  appropriate  degree  of  direction  as  the  Soldier  turns  his  body  in  that  direction  allowing  him  to 
navigate  on  a  magnetic  azimuth.  The  GPS,  when  activated,  displayed  the  Soldier’s  current 
location  on  a  mini  map  but  did  not  display  a  military  grid  reference  system  coordinate.  There 
currently  is  no  indication  or  capability  to  determine  the  distance  traveled  (e.g.,  pace  count). 

Shooting.  The  surrogate  weapons  available  replicate  what  an  Infantry  Squad  would  carry 
as  individual  weapons:  The  M4  Rifle;  the  M4/M320  Rifle/Grenade  Launcher;  and  the  M249 
Machine  Gun.  These  weapons  closely  replicated  the  actual  weapons  in  size,  weight,  and 
functionality.  Multiple  optics  are  available  for  each  weapon,  from  the  M68  Close  Combat  Optic 
(CCO)  to  the  Ml 50  Advanced  Combat  Optical  Gunsight  (ACOG)  to  the  AN/PAS- 13  Thermal 
Weapons  Sight  (TWS).  A  visual  facsimile  of  the  optic  is  displayed  in  the  HMD  when  a  pressure 
plate  located  in  the  buttstock  of  the  weapon  indicated  that  the  Soldier  placed  the  weapon  in  the 
pocket  of  the  shoulder  in  a  normal  firing  position. 

Soldiers  can  be  issued  a  virtual  basic  load  of  ammunition  for  each  system  and  required  to 
reload  once  all  rounds  had  been  expended  from  the  magazine,  drum,  or  tube.  Ballistic 
trajectories  for  each  type  of  round  were  incorporated  into  the  system  allowing  the  Soldiers  to 
apply  the  proper  lead  or  hold-off  when  engaging  moving  targets  or  targets  at  distance.  Rounds 
that  were  fired  are  visually  depicted  in  this  system.  Hits  result  in  wounding  or  killing  of  friendly 
and  enemy  personnel,  and  when  the  Soldier’s  avatar  was  killed,  the  HMD  went  black  and  the 
VSMM  shut  down. 

Audio  and  Soldier  communications.  Each  Soldier  wears  a  headset  and  microphone  that 
enabled  him  to  hear  voice,  radio,  and  battlefield  effects  (e.g.,  gunshots,  explosions,  ambient 
noise,  etc.).  Voice  communication  between  Soldiers  is  accomplished  via  the  microphone  with  no 
requirement  to  push  a  button.  The  communication  among  Soldiers  is  on  an  open  network,  and 
Soldiers  hear  all  communications  on  the  open  network  regardless  of  the  proximity  to  other 
Soldiers.  Radio  communication  between  teams  or  between  the  Squad  Leader  and  the  Platoon 
Leader  is  accomplished  with  push-to-talk  buttons  located  on  the  communications  junction  box  at 
the  front  of  the  sensor  harness.  Multiple  radio  channels  were  available  to  the  units  conducting 
training  to  replicate  actual  radio  capabilities. 
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Data  from  Dismounted  Soldier  Training  System  User  Assessment 


The  purpose  of  the  DSTS  User  Assessment  was  to  assess  the  training  eapabilities  of  the 
system  and  training  support  materials  prior  to  fielding.  The  User  Assessment  was  based  on  43 
performanee  requirements,  whieh  were  defined  by  the  Infantry  proponent  at  the  Maneuver 
Center  of  Exeellence  at  Fort  Benning.  These  performance  requirements  were  based  on  the 
realistic  execution  of  squad-level  infantry  tactical  tasks.  During  the  four-day  evaluation,  Soldiers 
received  DSTS  familiarization  training  and  then  completed  six  different  infantry  tactical 
scenarios  (offense,  defense,  and  reconnaissance  in  various  environments  including  urban, 
woodland,  and  desert).  Two  Army  Infantry  squads  plus  the  Platoon  Leader  and  Platoon  Sergeant 
participated  in  the  user  assessment. 

Even  though  much  of  the  User  Assessment  focused  on  the  technological  performance  of 
DSTS  (e.g.,  reliability  of  system,  battery  life,  and  network  performance)  and  the  sufficiency  of 
training  support  materials,  ARTs  focus  was  on  collecting  metrics  of  system  performance  that 
would  indicate  a  realistic  and  reliable  training  benefit.  Accordingly,  data-collection  instruments 
were  developed  to  record  Soldiers’  feedback  on  the  ability  to  move,  shoot,  and  communicate  in 
DSTS  and  to  utilize  DSTS  for  specific  training  outcomes.  All  research  data  was  aggregated 
across  missions  and  summarized  to  provide  input  to  the  User  Assessment. 

Method 

Participants.  Two  operational-unit  Infantry  squads  from  Fort  Benning  participated  in 
the  DSTS  User  Assessment.  Input  from  these  18  individuals  served  as  the  primary  data  for  the 
results  reported  here.  In  addition  to  these  two  nine-man  squads,  the  Platoon  Leader  and  Platoon 
Sergeant  from  the  participating  unit  had  roles  in  the  User  Assessment,  and  the  Platoon  Leader 
contributed  data  on  one  training-effectiveness  questionnaire. 

The  Platoon  Leader,  Platoon  Sergeant,  and  each  Squad  Leader  provided  background  data 
on  the  squads’  tactical  proficiency  and  training  experience.  These  perceptions  of  squad  tactical 
proficiency  indicated  the  frequency  and  recency  of  infantry  offensive  and  defensive  tasks 
(individual  and  collective),  the  type  of  environments  in  which  tasks  were  executed,  and  the 
ability  to  accomplish  doctrinal  proficiency  on  tasks.  This  data  as  well  as  individual  Soldier 
background  data  was  intended  to  provide  a  context  for  understanding  the  Soldiers’  responses  to 
DSTS  training.  That  is,  the  responses  Soldiers  provide  to  simulation  training  are  only  relevant  to 
the  degree  of  tactical  proficiency,  tactical  experience,  technological  proficiency,  and  experience 
with  simulation  (e.g.,  Allen,  Hays,  &  Buffardi,  1986;  Smith- Jentsch,  Jentsch,  Payne,  &  Salas, 
1999;  Waller,  2000). 

The  Squad  members  averaged  23-years  old  with  3  years  of  Infantry  experience,  39- 
months  time  in  service,  12-months  time  in  the  unit,  and  I  previous  deployment.  The  ranks 
ranged  from  Private  (E-2)  to  First  Lieutenant  (0-2).  All  Squad  members  reported  a  high  level  of 
confidence  with  using  computers  with  12  hours-per-week  average  computer  use.  Most  of  the 
computer  usage  was  reported  as  searching  the  internet.  In  addition,  55%  of  the  Squad  members 
reported  playing  computer  games  with  an  average  of  8  hours-per-week  of  game  playing.  Half 
(50%)  of  the  Squad  members  reported  prior  experience  with  simulation  training.  The  vast 
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majority  (i.e.,  90%)  of  Squad  members  believed  that  dismounted  simulation  training  would  be 
useful,  and  the  majority  (i.e.,  80%)  looked  forward  to  a  dismounted  infantry  simulator  training 
capability.  None  of  the  Squad  members  reported  color  blindness  or  prior  issues  with  motion 
sickness.  Appendix  A  presents  a  copy  of  the  instrument  used  to  collect  the  background  data. 

Although  the  training  experience  appeared  to  differ  slightly  between  squads,  both  squads 
had  frequently  trained  (i.e.,  3-5  times)  on  most  of  the  individual  tasks  and  collective  tasks  in  a 
live  training  environment  within  the  six  months  prior  to  the  DSTS  User  Assessment.  While  the 
squads  were  rated  as  “Trained”  or  “Practiced”  (ref  Department  of  Army,  1990)  on  all  of  the 
tasks,  the  squads  appeared  to  be  most  proficient  at  “movement”  tasks  as  compared  to  “shooting” 
tasks  or  “communication”  tasks.  The  complete  tactical-proficiency  ratings  are  given  in 
Appendix  B  (Platoon  Leader  and  Platoon  Sergeant)  and  Appendix  C  (Squad  Leaders). 

Materials.  Data-collection  instruments  were  developed  to  record  Soldiers’  feedback  on 
the  ability  to  move,  shoot,  and  communicate  in  DSTS  and  to  utilize  DSTS  for  specific  training 
outcomes.  The  primary  instrument  was  the  DSTS  Manned  Module  Capabilities  Checklist,  which 
assessed  Soldiers’  perceptions  of  their  abilities  to  execute  individual  tasks  and  collective  tasks  in 
the  simulation.  In  addition  to  the  capabilities  checklist,  post-training  questionnaires  assessed 
Leaders’  and  Soldiers’  perceptions  of  the  sufficiency  and  effectiveness  of  the  DSTS  training. 

DSTS  Manned  Module  Capabilities  Checklist.  This  instrument  was  used  to  record 
Soldier’s  perceptions  of  the  relative  ease  of  completing  physical  tasks  in  simulation  as  compared 
to  the  real  world.  Such  perceptions  are  sometimes  called  the  “realism”  or  “physical  fidelity”  of 
the  simulator/simulation.  In  the  case  of  the  DSTS  Manned  Module  Capabilities  Checklist,  the 
simulator  capabilities  focused  on  the  Soldiers’  abilities  to  move,  shoot,  communicate,  and 
navigate  in  the  virtual  environment.  The  individual  techniques  and  tasks  used  for  the  DSTS 
Manned  Module  Capabilities  Checklist  were  based  on  previous  assessment  research  on 
dismounted  infantry  simulation  (Pleban,  Dyer,  Salter,  &  Brown,  1998;  Salter,  Eakin,  &  Knerr, 
1999).  The  basic  inventory  of  dismounted  infantry  simulation  capabilities  from  these  previous 
assessments  was  augmented  and  refined  to  match  the  specific  performance  requirements  and 
characteristics  of  DSTS.  The  result  was  a  total  set  of  108  performance  capabilities. 

Each  of  the  performance  capabilities  assessed  in  the  DSTS  Manned  Module  Capabilities 
Checklist  was  rated  on  three  dimensions:  How  similar  the  task  was  to  perform  in  simulation  as 
compared  to  the  real  world;  how  quickly  the  task  was  to  perform  in  simulation  as  compared  to 
the  real  world;  and  how  dijficult  the  task  was  to  perform  in  simulation  as  compared  to  the  real 
world.  Each  of  the  three  dimensions  (i.e.,  similar,  quickly,  and  difficult)  was  rated  on  a  5-point 
scale  (Salter,  et  ah,  1999).  Eor  each  performance  capability,  there  were  also  response  options  to 
indicate  whether  or  not  a  data  collector  observed  the  task  being  conducted  during  training  and  to 
indicate  if  the  Soldiers  “performed,”  “did  not  perform,”  or  “could  not  perform”  the  task.  The 
purpose  of  these  response  options  was  to  be  sure  all  relevant  items  were  rated  by  the  Soldiers  and 
to  better  understand  why  items  may  not  have  been  rated.  Eigure  4  provides  an  example  of  DSTS 
Manned  Module  Capabilities  Checklist  items  and  format,  and  the  entire  instrument  is  provided  in 
Appendix  D. 
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Figure  4.  Example  of  DSTS  Manned  Module  Capabilities  Checklist  Items. 

Squad  Member  Post-Training  Questionnaire,  The  Squad  Member  Post-Training 
Questionnaire  assessed  Soldiers’  post-training  perceptions  of  DSTS  training  sufficiency.  The 
questionnaire  contained  items  to  assess  the  sufficiency  of  training  support  materials,  training 
support  personnel,  and  training  time.  These  items  were  rated  on  a  5-point  scale  with  anchors  at 
“very  sufficient”  and  “very  insufficient.”  Free-response  items  were  used  to  obtain  additional 
details  on  select  items.  Responses  were  also  provided  for  Soldiers  to  indicate  if  a  given  training 
feature  was  not  encountered. 

The  questionnaire  also  contained  items  to  assess  the  training  sufficiency  of  scenarios  used 
and  to  gather  Soldiers’  opinions  on  the  fielding  of  DSTS.  Both  of  these  sets  of  items  were  rated 
using  a  5-point  scale.  The  scenarios  items  were  rated  on  the  same  5-point  scale  as  the  other 
training  items  (i.e.,  “very  sufficient”  to  “very  insufficient”),  while  the  fielding  items  were  rated 
on  a  5-point  scale  with  anchors  at  “strongly  agree”  and  “strongly  disagree.”  Finally,  the 
questionnaire  contained  four  free-response  items  to  address  Soldiers’  overall  assessments  of 
DSTS  training  effectiveness.  The  entire  instrument  is  presented  in  Appendix  E. 

Leader  Post-Training  Questionnaire,  This  instrument  was  used  to  assess  Platoon 
Leaders’  perceptions  of  DSTS  training  effectiveness  across  a  variety  of  dimensions.  The 
dimensions  included  training  support  materials,  system  support  personnel,  the  ability  to  conduct 
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Leader  pre -mission  planning,  and  the  ability  to  execute  Squad  collective  tasks  in  offense 
missions  and  defense  missions.  Responses  were  given  on  5-point  scales  for  “effectiveness”  or 
“sufficiency”  depending  on  the  item.  There  was  also  the  response  option  to  indicate  that  the 
respondent  did  not  have  the  opportunity  to  observe  or  participate  in  a  given  dimension  or  task. 
The  items  on  the  Leader  Post-Training  Questionnaire  were  derived  from  unit- training 
management  directives  and  unit  mission  essential  task  lists  from  Army  Doctrinal  Publication  7.0, 
Training  Units  and  Developing  Leaders  (Department  of  Army,  2012),  and  Training  Circular  3- 
21.8,  Infantry  Rifle  and  Mechanized  Platoon  Collective  Task  Publication  (Department  of  the 
Army,  2013).  The  entire  instrument  is  presented  in  Appendix  F. 

Procedure.  Prior  to  the  start  of  the  DSTS  User  Assessment,  research  personnel  (the  first 
and  third  author)  met  with  the  participating  units  at  their  respective  units.  The  purpose  of  the 
data  collection  was  described  and  informed  consent  was  obtained.  Participants  then  completed 
background  questionnaires.  The  Platoon  Leader  and  Platoon  Sergeant  also  completed  tactical 
proficiency  and  training  experience  questionnaires  for  each  squad. 

The  DSTS  User  Assessment  was  conducted  over  four  days  at  Fort  Benning,  GA.  At  the 
outset  of  the  User  Assessment,  both  participating  nine-man  Army  squads  were  given  a 
familiarity  brief  about  DSTS  and  then  fitted  into  the  wearable  manned-module  equipment.  The 
squads  were  initially  given  a  familiarization  scenario.  This  scenario  was  individually  completed 
by  all  squad  members  and  the  purpose  was  to  allow  each  person  to  practice  movements  and 
interactions  in  the  virtual  environment.  Three  additional  personnel  trained  on  the  operation  of 
the  MFWS  during  this  “train-up”  period. 

During  the  formal  assessment  exercises,  each  squad  separately  completed  offense, 
defense,  and  area  reconnaissance  missions  in  desert,  woodland,  and  urban  virtual  environments. 
Each  mission  was  designed  to  take  45  -  60  minutes  to  complete.  After  the  completion  of  each 
mission,  the  squad  removed  the  manned  modules  and  participated  in  a  mission  AAR.  There  was 
a  break  period  between  each  mission,  and  the  squads  completed  two  or  three  missions  per  day. 
Other  missions  were  executed  to  test  technical  capabilities  of  the  system  but  were  not  part  of  data 
presented  in  this  report. 

After  the  final  mission  each  day,  participants  worked  with  data  collectors  to  provide  input 
on  the  DSTS  Manned  Module  Capabilities  Checklist.  The  data  collectors  were  each  retired 
senior  non-commissioned  officers  with  extensive  experience  collecting  interview  data  for  ART 
The  participants  were  separated  into  three  groups.  The  two  Squad  Leaders  formed  one  group 
and  the  squad  members  from  each  respective  squad  formed  the  other  two  groups.  The  Squad 
Leaders  participated  separate  from  their  squads  in  order  to  facilitate  more  input  from  junior 
squad  members.  In  each  group,  participants  took  turns  providing  ratings  on  each  system 
capability  for  each  type  of  rating  (i.e..  Similarity,  Quickness,  and  Difficulty).  The  rating  began 
with  the  data  collector  reading  the  capability  to  the  group  and  the  group  indicting  whether  or  not 
the  capability  was  performed  during  the  day’s  missions.  Each  group  member,  in  turn,  verbally 
provided  a  rating  on  the  respective  scale  and  the  data  collector  recorded  the  rating  on  the  DSTS 
Manned  Module  Capabilities  Checklist.  The  group  members  alternated  giving  the  initial 
response  on  ratings  in  order  to  reduce  demand  characteristics  as  much  as  possible.  Once  ratings 
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were  given  on  all  three  types  of  ratings  for  one  capability,  the  next  capability  was  read  and  the 
procedure  repeated  until  all  items  on  DSTS  Manned  Module  Capabilities  Checklist  were  rated. 

At  the  completion  of  User  Assessment,  the  participants  were  asked  to  complete  the  post¬ 
training  questionnaires.  All  participants  were  taken  to  a  classroom,  given  instructions  for  the 
questionnaires,  and  allowed  to  individually  complete  the  questionnaires  at  their  own  pace.  Once 
all  questionnaires  were  completed,  the  participants  were  given  a  short  debrief  and  thanked. 

Results 

DSTS  Manned  Module  Capabilities  Checklist,  The  108  performance  capabilities 
assessed  on  the  DSTS  Manned  Module  Capabilities  Checklist  were  crosswalked  to  the  43  system 
requirements  provided  by  the  DSTS  proponent  at  the  Maneuver  Center  of  Excellence.  The  raw 
ratings  were  aggregated  according  to  the  crosswalk  into  43  performance  capabilities  matched  to 
the  43  system  requirements.  The  ratings  on  the  resulting  43  performance  capabilities  were 
further  aggregated  across  all  participants  and  across  all  missions  for  analysis.  For  analysis,  the 
ratings  were  also  converted  from  the  original  5-point  scale  used  to  obtain  the  data  to  a  3-point 
scale  for  analysis.  The  conversion  was  made  because  few  extreme  ratings  were  given  and 
because  the  results  needed  to  be  categorized  to  support  clear  decisions  for  the  DSTS  User 
Assessment.  That  is,  the  DSTS  User  Assessment  stakeholders  wanted  to  assign  a  “Green,” 
“Amber,”  or  “Red”  decision  to  each  DSTS  system  capability  to  indicate  a  given  capability  was 
ready,  acceptable,  or  unacceptable  to  train  Soldiers.  The  Green-Amber-Red  convention  was 
adopted  in  the  present  report  for  ease  of  interpretation. 

The  scale  conversion  involved  aggregating  response  anchors  that  indicted  readiness  into 
the  “Green”  category  (e.g.,  “Exactly  Eike”  and  “Very  Similar”  responses  on  the  Similarity 
rating),  assigning  the  middle  response  anchor  to  the  “Amber”  category  (e.g.,  “Somewhat 
Similar”  response  on  the  Similarity  rating),  and  aggregating  the  response  anchors  that  indicated 
unacceptable  performance  into  the  “Red”  category  (e.g.,  “Very  Different”  and  “Completely 
Different”  responses  on  the  Similarity  rating).  The  cumulative  frequencies  for  each  Green- 
Amber-Red  category  on  each  type  of  rating  (i.e..  Similarity,  Quickness,  and  Difficulty)  were 
used  to  analyze  each  DSTS  capability  rated  on  the  DSTS  Manned  Module  Capabilities  Checklist. 

The  first  step  in  interpreting  the  DSTS  Manned  Module  Capabilities  Checklist  ratings 
was  to  conduct  correlation  analyses  on  the  Green  responses  across  each  type  of  rating  (e.g.. 
Similarity,  Quickly,  and  Difficulty).  This  analysis  ostensibly  indicated  whether  there  was  a 
single  dimension  influencing  more  than  one  type  of  rating.  If  any  two  types  of  ratings  exhibited 
a  high  level  of  dependence  across  all  system  capabilities,  then  it  could  be  concluded  that  those 
ratings  were  capturing  the  same  impact  on  Soldier  performance.  Only  the  Green  ratings  were 
used  for  this  analysis  because  the  criteria  for  responses  indicating  system  “success”  would  be 
more  strict,  and  therefore  more  reliable,  than  responses  indicating  system  difficulties.  The 
resulting  Spearman  correlations  were  p  =  .95  (Similarity  -  Quickness),  p  =  .79  (Similarity  - 
Difficulty),  and  yO  =  .84  (Quickness  -  Difficulty).  Even  though  all  correlations  were  significant  at 
the  .05  alpha  level,  the  pattern  of  correlations  clearly  indicated  that  Similarity  ratings  were  most 
similar  to  Quickness  ratings  and  that  Similarity  ratings  were  most  divergent  from  Difficulty 
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ratings.  As  a  result,  subsequent  data  interpretation  focused  only  on  Similarity  ratings  and 
Difficulty  ratings. 

The  next  step  in  interpreting  the  DSTS  Manned  Module  Capabilities  Checklist  ratings 
was  to  classify  each  of  the  43  performance  capabilities  into  Green,  Amber,  Red,  or 
“Undetermined.”  This  classification  indicated  the  degree  to  which  DSTS  was  perceived  to 
provide  effective  training.  Non-parametric  analyses  of  rating  frequencies  were  conducted  to 
determine  the  patterns  of  ratings  within  each  performance  capability  and  for  each  rating  (i.e.. 
Similarity  and  Difficulty).  A  given  performance  capability  was  categorized  as  “Green”  if  the 
statistically  highest  frequency  of  responses  were  in  the  Green  rating  category.  For  example,  if 
the  statistical  majority  of  Similarity  ratings  for  a  given  performance  capability  were  in  the 
“Exactly  Like”  response  category,  then  the  capability  was  classified  as  Green.  Likewise,  those 
performance  capabilities  most  frequently  rated  as  “Somewhat  Similar”  were  classified  as  Amber, 
and  those  performance  capabilities  most  frequently  rated  as  “Completely  Different”  were 
classified  as  Red.  If  there  were  no  significant  differences  among  rating  frequencies  for  a  given 
performance  capability,  then  the  capability  was  categorized  as  “Undetermined.”  This  category 
simply  meant  that  there  were  no  consistent  ratings  across  Soldiers  and  missions  and  that  the 
training  impact  of  the  performance  capability  could  not  be  determined.  Each  performance 
capability  was  likewise  categorized  on  Difficulty  ratings. 

The  performance  capabilities  were  first  classified  based  on  the  ratings  of  Similarity.  By 
identifying  how  many  performance  capabilities  were  classified  in  each  type  of  rating  (i.e..  Green, 
Amber,  and  Red),  it  was  possible  to  determine  the  sufficiency  of  DSTS  to  emulate  critical 
elements  of  Infantry  tasks.  Each  performance  capability  was  then  classified  by  its  ratings  of 
Difficulty  (i.e.,  “More  Difficult,”  “About  the  Same,”  and  “Less  Difficult”).  Two  performance 
capabilities  were  not  performed  during  the  User  Assessment  and  had  no  data  associated  with 
them:  Interact  with  Civilian;  and  Disperse  a  Crowd  of  Civilians.  The  response  frequencies  for 
the  ratings  of  the  remaining  41  performance  capabilities  are  given  in  Table  1  and  Table  2.  The 
tables  also  present  the  statistical  test  (i.e.,  chi-square)  for  the  frequencies. 

Table  1  organizes  the  performance  capabilities  by  their  categorization  on  Similarity 
ratings  (i.e..  Green- Amber-Red-Undetermined).  Twenty  performance  capabilities  (48.78%) 
were  classified  as  Green  on  Similarity  ratings.  This  pattern  of  results  indicated  that  Soldiers 
were  able  to  perform  Infantry  tasks  in  DSTS  in  a  manner  that  would  not  distract  from  training. 
The  next  largest  category  of  Similarity  ratings  was  Undetermined.  Thirteen  (31.71%) 
performance  capabilities  did  not  have  a  statistical  difference  among  the  frequencies  of  ratings. 
This  result  indicated  that  either  Soldiers  did  not  agree  about  the  how  the  capability  was 
performed  relative  to  real  life  or  that  the  capability  differed  across  missions.  Regardless  of  the 
reasons  for  the  ratings,  these  performance  capabilities  may  need  additional  specification  in 
subsequent  refinement  of  DSTS.  Only  six  (14.43%)  performance  capabilities  were  categorized 
as  Amber  on  Similarity,  and  only  two  (4.88%)  capabilities  were  categorized  as  Red. 
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Table  1 

Response  Frequencies  for  DSTS  Manned  Module  Capabilities  Checklist  Similarity  Ratings. 


Similarity  Ratings 

Exactly 

Somewhat 

Completely 

Performance  Capability 

Like 

Similar 

Different 

IL 

Green 

Conduct  Platoon  movement 
techniques 

175 

136 

65 

49.63 

Move  over,  through,  around 
obstacle 

157 

114 

56 

47.12 

Move  using  cover 

178 

114 

61 

58.03 

Conduct  tactical  movement  to 
an  Assault  Position 

227 

179 

117 

34.88 

Move  under  direct  fire 

322 

268 

157 

62.13 

React  to  indirect  fire  mounted 
and  dismounted 

188 

126 

101 

29 

Conduct  movement  during 
limited  visibility 

201 

156 

81 

50.34 

Enter  a  Building  in  Urban  Area 

317 

259 

155 

55.29 

Move  to  a  Tactical  Assembly 

Area  or  designated  area  short  of 
the  Defense  Position 

247 

213 

143 

27.98 

Assign  primary,  alternate,  and 
supplementary  Fighting 

Positions 

173 

115 

57 

58.5 

Select  covered  and  concealed 
routes  between  primary, 
alternate,  and  supplementary 
Defense  Positions 

164 

107 

56 

53.55 

Tied  in  with  unit  left  and  right 

186 

127 

64 

140.95 

Conduct  a  squad  combat  patrol. 

348 

303 

168 

64.28 

Requests  direct  and/or  indirect 
fires 

68 

52 

38 

8.55 

Use  Armored  Vehicle  to 

provide  cover  fires  to  Assault  a 
building  and  clear  a  building 

46 

9 

13 

36.38 

Use  Armored  Vehicle  to  breach 
walls 

40 

5 

12 

36.09 

13 


Table  1  (continued) 


Similarity  Ratings 


Exactly 

Somewhat 

Completely 

Performance  Capability 

Like 

Similar 

Different 

'I 

Green 

Destroy  a  position  in  a  building 
with  Armored  Vehiele  direet 

23 

2 

13 

17.41 

fire 

Use  vehiele  to  seeure  eleared 

28 

A 

29.89 

portions  of  the  Objeetive 

0 

Clear  fields  of  fire 

63 

9 

2 

90.37 

Use  Armored  Vehiele  to 
provide  eover  dismounted 
movement. 

30 

5 

8 

26.04 

Engage  multiple  MG  targets 

405 

422 

222 

70.33 

Amber 

Detect,  Suppress,  Destroy, 
enemy  Anti-Armor  position 

126 

129 

86 

10.13 

Identify  dead  space  in  sector 

115 

117 

44 

37.58 

Place  obstacles  concealed  from 
enemy  observation 

146 

150 

86 

20.18 

Engage  targets  with  M249  SAW 

270 

284 

148 

47.82 

Engage  targets  with  M4/M4A1 
Carbine. 

270 

284 

148 

47.82 

Red 

Employ  claymore  mines  and 
other  obstacles 

0 

2 

18 

30.12 

Undetermined 

Conduct  a  Breach  Urban 

2 

5 

6 

1.99 

Armored  Eighting  Vehicle 

Carries  soldiers  to  dismount 

4 

4 

10 

2 

Evacuate  casualties  to  a  safe 
location 

10 

10 

11 

0.06 

Platoon  occupies  Assault 

Position 

72 

77 

78 

0.232 

Select  temporary  Eighting 

Position 

7 

7 

3 

1.88 

14 


Table  1  (continued) 


Similarity  Ratings 


Performance  Capability 

Exactly 

Like 

Somewhat 

Similar 

Completely 

Different 

IL 

Undetermined 

React  to  lED  while  dismounted 

2 

6 

3 

2.36 

Send  Situation  Reports  to  higher 
headquarters 

6 

5 

10 

2 

Perform  voice  communications 

6 

5 

9 

1.29 

Use  visual  signaling  technique 

0 

2 

2 

1.98 

Employ  smoke 

0 

3 

1 

3.48 

Engage  targets  with  Tank  main 
gun 

11 

2 

14 

8.67 

Interact  with  civilian 

0 

0 

0 

0 

Disperse  a  crowd  of  civilians. 

0 

0 

1 

1.93 

Move  through  a  crowd  of 
civilians. 

2 

2 

0 

1.98 

Conduct  Breach  Assault 

Use  vehicle  to  establish  a  Road 

4 

1 

2 

1.98 

Block  or  barricade  to  isolate 
Objective  or  building 

4 

4 

10 

3.98 

Table  2  organizes  the  performanee  eapabilities  by  their  of  eategorization  Diffieulty 
ratings.  Twenty-five  (61%)  performanee  eapabilities  were  classified  as  Amber  on  Difficulty 
ratings.  This  finding  indicated  that  performance  of  most  Infantry  tasks  in  DSTS  were 
appropriately  difficult  irrespective  of  how  similar  the  task  was  to  perform.  Only  three  (7.32%) 
performance  capabilities  were  categorized  as  Green  on  Difficulty  ratings.  These  capabilities 
were  likely  rated  as  “Less  Difficult”  because  Soldiers  only  need  to  “click”  on  a  button  to  perform 
the  task  rather  than  use  complex  procedures  or  physical  exertion.  For  example,  evacuating 
casualties  in  DSTS  only  requires  that  the  Soldier’s  avatar  “walk  over”  the  casualty  and  then  walk 
away  with  no  physical  effort  required.  There  were  seven  (17.07%)  performance  capabilities 
categorized  as  Undetermined  on  Difficulty  ratings.  Six  (14.63%)  performance  capabilities  were 
not  rated  for  Difficulty  because  they  were  vehicle-specific  capabilities  and,  even  though  Soldiers 
could  rate  the  Similarity  of  interaction  with  vehicles.  Soldiers  could  not  rate  the  Difficulty  of 
positioning  vehicles. 
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Table  2 

Response  Frequencies  for  DSTS  Manned  Module  Capabilities  Checklist  Difficulty  Ratings. 


Difficulty  Ratings 


Performance  Capability 

Less 

Difficult 

About  the 
Same 

More 

Difficult 

Green 

Employ  claymore  mines  and 
other  obstacles 

17 

1 

2 

24.10 

Conduct  a  Breach  Urban 

9 

4 

1 

7.00 

Evacuate  casualties  to  a  safe 

22 

f. 

a 

20.19 

location 

o 

Amber 

Conduct  Platoon  movement 
techniques 

21 

233 

113 

184.76 

Move  over,  through,  around 
obstacle 

15 

223 

78 

216.01 

Move  using  cover 

15 

248 

82 

171.98 

Conduct  tactical  movement  to 
an  Assault  Position 

91 

305 

128 

150.08 

Move  under  direct  fire 

100 

443 

191 

258.09 

React  to  indirect  fire  mounted 
and  dismounted 

55 

242 

no 

136.18 

Conduct  movement  during 
limited  visibility 

27 

287 

125 

235.64 

Enter  a  Building  in  Urban  Area 

108 

434 

176 

247.16 

Move  to  a  Tactical  Assembly 
Area  or  designated  area  short  of 
the  Defense  Position 

99 

337 

155 

67.65 

Assign  primary,  alternate,  and 
supplementary  Eighting 

Positions 

15 

241 

81 

240.45 

Select  covered  and  cancelled 
routes  between  primary, 
alternate,  and  supplementary 
Defense  Positions 

15 

226 

78 

221.24 

Tied  in  with  unit  left  and  right 

24 

260 

85 

233.44 

Conduct  a  squad  combat  patrol. 

115 

402 

217 

173.02 

16 


Table  2  (continued) 


Difficulty  Ratings 


Performance  Capability 

Less 

Difficult 

About  the 
Same 

More 

Difficult 

Amber 

Requests  direet  and/or  indireet 
fires 

22 

78 

52 

31.00 

Engage  multiple  MG  targets 

591 

591 

276 

136.11 

Deteet,  Suppress,  Destroy, 
enemy  Anti-Armor  position 

88 

150 

72 

32.85 

Identify  dead  spaee  in  seetor 

10 

9 

5 

8.71 

Plaee  obsateles  eoneealed  from 
enemy  observation 

65 

202 

94 

91.12 

Engage  targets  with  M249 

SAW 

114 

394 

184 

184.10 

Engage  targets  with  M4  M4A1 
Carbine. 

114 

394 

184 

184.10 

Armored  Eighting  Vehiele 
Carries  soldiers  to  dismount 

6 

6 

0 

6.00 

Platoon  oeeupies  Assault 

Position 

78 

104 

42 

25.96 

Seleet  temporary  Eighting 
Position 

10 

11 

6 

10.71 

Reaet  to  lED  while  dismounted 

0 

10 

1 

16.55 

Move  through  a  erowd  of 
eivilians. 

0 

4 

0 

8.00 

Undetermined 

Send  Situation  Reports  to 
higher  headquarters 

11 

6 

3 

4.90 

Perform  voiee  eommunieations 

11 

6 

3 

4.90 

Use  visual  signaling  teeh 

3 

0 

1 

3.50 

Employ  smoke 

3 

0 

1 

3.50 

Conduet  Breaeh  Assault 

4 

3 

1 

1.75 

Clear  fields  of  fire 

0 

0 

1 

2.00 

17 


Table  2  (continued) 


Performance  Capability 


Difficulty  Ratings 

Less  About  the  More 
Difficult  Same  Difficult 


Undetermined 

Use  Armored  Vehicle  to 

provide  cover  dismounted  5 

movement. 

Not  Rated 

Use  vehicle  to  establish  a  Road 
Block  or  barricade  to  isolate  0 

Objective  or  building 

Use  Armored  Vehicle  to 

provide  cover  fires  to  Assault  a  0 

building  and  clear  a  building 

Use  Armored  Vehicle  to  breach  „ 

walls 

Destroy  a  position  in  a  building 

with  Armored  Vehicle  direct  0 

fire 

Engage  targets  with  Tank  main  ^ 

gun 

Use  vehicle  to  secure  cleared  ^ 

portions  of  the  Objective 


3 


0 

0 

0 

0 

0 

0 


1  2.67 


0 

0 

0 

0 

0 

0 


The  frequencies  for  Quickness  ratings  are  provided  in  Appendix  G  ordered  by  Green- 
Amber-Red-Undetermined  categories.  These  frequencies  were  provided  in  order  to  present  a 
complete  record  of  the  data. 

The  final  step  in  interpreting  the  DSTS  Manned  Module  Capabilities  Checklist  ratings 
was  to  determine  if  Soldiers’  ratings  of  the  performance  capabilities  changed  over  time.  If 
ratings  changed  over  the  four  days  of  the  User  Assessment,  then  some  inference  can  be  made 
about  the  impact  of  experience  on  DSTS  training.  For  example,  if  ratings  were  generally  more 
favorable  over  the  course  of  the  User  Assessment,  then  it  might  be  inferred  that  some  of  the 
perceived  difficulties  using  DSTS  could  be  mitigated  with  experience.  For  this  analysis,  the 
proportion  of  Similarity  ratings  and  Difficulty  ratings  in  the  Red  category  were  compared  across 
each  day  of  the  user  assessment.  The  Red  category  was  chosen  for  this  analysis  because  it 
represent  impediments  to  training,  and  should  be  more  sensitive  to  training  experience.  The 
proportions  of  Red  category  frequencies  are  presented  in  Figure  5.  Accordingly,  the  proportion 
of  Red  ratings  significantly  decreases  (^^(3)  =  10.63)  to  16%  on  the  final  day  of  the  User 
Assessment  from  a  high  of  34%  on  the  second  day.  The  reduction  in  the  proportion  of  Red 
ratings  on  the  final  day  of  the  User  Assessment  indicated  that  it  may  require  8-10  hours  of 
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structured  familiarization  training  before  Soldiers  were  comfortable  enough  with  DSTS  to 
effectively  exeeute  training. 


© 

c 


p 

£ 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


Day 


Figure  5.  Proportion  of  Combined  Red-Category  Ratings  for  Eaeh  Day  of  the  User  Assessment. 

Squad  Member  Post-Training  Questionnaire.  The  eumulative  response  frequeneies 
for  each  item  of  the  Squad  Member  Post-Training  Questionnaire  are  embedded  in  Appendix  E 
along  with  representative  free-response  comments.  The  response  frequencies  indicated  that  25% 
of  Soldiers  believed  DSTS  caused  distractions  and  contributed  to  bad  training  habits  but  that 
65%  of  Soldiers  felt  immersed  and  challenged  in  DSTS.  The  questionnaire  also  indieated  that 
45%  of  Soldiers  thought  that  the  audio  cues  in  DSTS  were  not  effective  in  locating  significant 
actions  but  that  only  15%  of  Soldiers  thought  that  visual  cues  were  not  effeetive  for  loeating 
significant  actions.  Einally,  the  responses  on  the  Squad  Member  Post-Training  Questionnaire 
indieated  that  47%  of  Soldiers  believed  DSTS  was  ready  to  field  to  the  Army,  28%  were  not  sure 
about  the  readiness  to  field,  and  25%  believed  DSTS  was  not  ready  to  field. 


Leader  Post-Training  Questionnaire.  The  responses  given  on  the  header  Post-Training 
Questionnaire  are  embedded  in  Appendix  E.  Because  there  was  only  one  respondent  to  this 
questionnaire  (i.e.,  the  Platoon  Leader),  it  is  difficult  to  make  inferences  about  the  sufficiency  of 
DSTS.  However,  the  Platoon  Leader’s  responses  clearly  indicated  that  DSTS  could  support  the 
evaluation  of  unit  performanee  on  troop  leading  proeedures  and  on  various  mission  executions 
(e.g.,  attack  and  defend).  With  regard  to  the  transfer  of  skills  to  a  real-world  setting,  the  Platoon 
Leader  indicated  that  the  virtual  environment  and  DSTS  hardware  somewhat  helped  with  transfer 
because  the  system  makes  operational  functions  better.  However,  the  Platoon  Leader’s 
responses  showed  reservation  that  individual  task  performanee  improvements  were  possible. 

The  Platoon  Leader’s  perceptions  of  the  training  support  materials  and  of  the  readiness  to  field 
were  generally  positive,  whieh  eehoed  the  perceptions  of  the  squad  members. 
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Summary  of  User  Assessment  Data.  Taken  together,  the  data  from  the  User 
Assessment  indieated  that  there  was  a  generally  positive  pereeption  of  the  training  sufficieney  of 
DSTS.  Ratings  on  the  performanee  eapabilities  were  mostly  elassified  as  aeceptable  (i.e.,  Amber 
or  Green)  for  both  the  similarity  of  performanee  in  DSTS  and  the  diffieulty  of  performanee  in 
DSTS.  Moreover,  performanee  eapabilities  ratings  eategorized  as  unaeeeptable  (i.e..  Red) 
deereased  with  more  experienee  in  DSTS.  However,  there  were  some  limitations  noted  about  the 
performance  capabilities  that  were  assessed.  More  specifically,  some  capabilities  were  not 
complete,  and  some  capabilities  were  not  present  to  fully  execute  Infantry  Squad  tasks.  Table  3 
lists  a  set  of  performance  capability  limitations  based  on  observations  of  data  collectors  and 
comments  by  User  Assessment  participants. 

Table  3 

Observed  Capability  Limitations  and  Non-existent  Capabilities  for  DSTS. 


Capability  Limitations 

•  Soldier’s  ability  to  communicate  via  voice  and  radio 

•  Weapons/ Ammunition  must  be  pre-loaded  in  the  Soldier’s  inventory  to  allow  the 
Soldiers  to  select/emplace/recover/detonate 

•  Objects  placed  within  the  scenario  must  have  the  functionality  for  the  Soldier  to 
interact  with  them  (e.g.,  fences  or  buildings  that  are  expected  to  be  breached) 

•  Characters  that  are  expected  to  interact  with  Soldiers  must  be  designated  role 
players  using  Multi-Functional  Work  Stations 

Non-existent  capabilities 

•  Soldiers  ability  to  roll  left  or  right 

•  Soldiers  ability  to  select  and  view  a  map,  either  full  scale  or  mini 

•  Squad  Leaders  ability  select  and  view  binoculars 

•  Squads  ability  to  drop  trees 

•  Soldiers  ability  to  treat  casualties  {can  only  carry  or  drag) 

•  Soldiers  ability  to  lean  left  or  right 

•  Soldiers  ability  to  perform  hand  and  arm  signals 

•  Soldiers  ability  to  activate  laser  (PEQ-2  or  PEQ-15) 

•  Soldiers  ability  to  determine  distance  traveled 

•  Soldier  ability  to  select  and  view  a  GPS  {The  current  view  is  only  a  map  that  has 
a  MGRS  grid  superimposed  on  it,  it  does  not  give  the  user  a  MGRS  6-,  8-,  or  10- 
digit  grid  coordinate.) 


Notes:  PEQ  =  portable  laser  combination;  GPS  =  global  positioning  system;  MGRS  =  military 
grid  reference  system. 


20 


Data  from  Joint  Forces  Bold  Quest  2012 


Bold  Quest  is  a  warfighting  experiment  eondueted  by  Joint  Forees  Command.  In  2012, 
Bold  Quest  was  eondueted  in  conjunetion  with  the  Army  Expeditionary  Warrior  Experiment  held 
at  Eort  Benning,  GA  and  focused  on  immersive  training  effectiveness  for  the  small  unit  and 
small  unit  leaders.  Several  virtual  immersive  technologies  were  assessed  during  Bold  Quest 
2012  including  marksmanship  trainers,  game-based  trainers,  mission  rehearsal  tools,  and 
dismounted  infantry  simulators  (e.g.,  DSTS).  The  Bold  Quest  2012  experiment  was  based  on  the 
comparison  of  training  in  the  virtual  environment  to  training  in  the  live  environment  and  on  the 
impact  of  virtual  training  on  live  performance.  As  such.  Bold  Quest  2012  presented  an 
opportunity  for  a  more  in-depth  examination  of  the  training  sufficiency  of  DSTS. 

Whereas  the  purpose  of  DSTS  User  Assessment  was  mostly  to  determine  the  physical 
verisimilitude  and  functional  verisimilitude  of  the  system  capabilities.  Bold  Quest  2012  was 
more  focused  on  training  outcomes.  Eike  the  User  Assessment,  Bold  Quest  2012  used  Soldier 
input  to  determine  those  training  outcomes.  Soldiers  again  rated  DSTS  performance  capabilities, 
and,  in  addition.  Soldiers  provided  input  on  the  impact  DSTS  training  had  on  decision  making, 
preparation  for  live  execution,  and  operational  realism.  At  the  heart  of  the  Bold  Quest  2012 
assessment  was  the  comparison  of  virtual  training  to  live  training.  The  assessment  consisted  of 
two  groups  of  Soldiers.  Each  group  executed  a  live  blank-fire  mission  followed  by  mission- 
specific  training  and  a  final  execution  of  the  same  mission.  One  group  of  Soldiers  trained  for  the 
final  mission  execution  in  the  virtual  environment  (i.e.,  DSTS)  and  the  other  group  trained  in  the 
live  environment.  By  doing  so,  comparisons  of  virtual  training  to  live  training  were  possible  as 
were  comparisons  virtual  mission  execution  to  live  mission  execution. 

The  entire  Bold  Quest  2012  Dismounted  Infantry  simulator  experiment  was  more 
extensive  than  is  presented  in  this  report.  Eirst,  Bold  Quest  2012  included  two  immersive 
dismounted  infantry  simulations:  DSTS  and  Small  Unit  Virtual  Immersion  System  (VIRTSIM). 
VIRTSIM  utilizes  motion-capture  technology  to  translate  individual  movements  in  three- 
dimensional  space  into  movements  in  the  virtual  environment.  Second,  three  types  of  Infantry 
Squads  participated  in  the  experiment:  Two  9-man  U.S.  Army  squads;  one  8-man  Royal 
Canadian  Regiment  squad;  and  one  13-man  U.S.  Marine  Corps  squad.  Einally,  three  types  of 
missions  were  executed  across  simulators  and  units:  Cordon  and  Search;  Area  Reconnaissance; 
and  Attack  an  Urban  Area.  However,  because  the  focus  of  this  report  is  on  the  training 
sufficiency  of  DSTS  for  the  U.S.  Army,  only  the  data  provided  by  the  U.S.  Army  squads  for 
DSTS  are  presented  here. 

Method 

Participants.  Two  squads  from  the  Experimental  Eorce  at  Eort  Benning  provided  data 
during  Bold  Quest  2012.  The  Experimental  Eorce  supports  the  U.S.  Army  Maneuver  Battle  Eab 
to  conduct  tests  and  experiments  on  new  technologies,  equipment,  and  concepts.  The  Platoon 
Eeader  and  Platoon  Sergeant  for  the  two  squads  also  participated  in  the  assessment,  and  the  total 
number  of  participants  was  20.  The  average  age  of  the  participants  was  24  years-old  (range:  19  - 
39),  and  the  average  time  in  services  was  4  years  (range:  1-16).  Half  of  the  participants  (i.e. 

10)  had  previously  deployed  for  an  average  of  22  total  months,  but  none  of  the  Soldiers  in  the 
Experimental  Eorce  had  previously  deployed  together.  The  ranks  ranged  from  Private  (E-2)  to 
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First  Lieutenant  (0-2).  All  Squad  members  reported  a  high  level  of  eonfidenee  with  using 
eomputers  with  14  hours-per-week  average  computer  use.  Most  of  the  computer  usage  was 
reported  as  searching  the  internet.  In  addition,  63%  of  the  Squad  members  reported  playing 
computer  games  with  an  average  of  8  hours-per-week  of  game  playing.  More  than  half  (75%)  of 
the  Squad  members  reported  prior  experience  with  simulation  training. 

Materials.  As  with  the  User  Assessment,  the  DSTS  system  performance  capabilities 
were  assessed  with  a  checklist.  Likewise,  Soldier  responses  were  collected  on  a  number  of 
questionnaires.  These  questionnaires  captured  Soldiers’  perceptions  of  specific  training 
environment  characteristics  and  of  training  outcomes. 

DSTS  Performance  Capabilities  Checklist.  This  instrument  was  a  modified  version  of 
the  DSTS  Manned  Module  Capabilities  Checklist  used  for  the  User  Assessment.  As  with  the 
DSTS  Manned  Module  Capabilities  Checklist,  the  DSTS  Performance  Capabilities  Checklist 
asked  Soldiers  to  rate  how  similar  and  how  difficult  tasks  were  to  complete  in  simulation  as 
compared  to  the  real  world.  However,  because  ratings  for  “quickly”  were  highly  correlated  with 
ratings  for  “similar”  on  the  DSTS  Manned  Module  Capabilities  Checklist,  the  “quickly”  ratings 
were  replaced  with  “accurately”  ratings  on  the  DSTS  System  Capabilities  Checklist. 

“Accurately”  ratings  asked  Soldiers  to  rate  how  accurately  a  task  could  be  completed  in 
simulation  as  compared  to  the  real  world.  Each  of  these  ratings  (i.e.,  similar,  difficult,  and 
accurately)  were  rated  on  a  3-point  scale  as  opposed  to  5-point  scales  used  for  the  DSTS  Manned 
Module  Capabilities  Checklist.  The  anchors  for  each  rating  scale  were  aligned  to  the  “Green,” 
“Amber,”  and  “Red”  categories  used  to  interpret  the  results  of  the  User  Assessment  data.  That 
is,  each  scale  point  represented  ready,  acceptable,  or  unacceptable.  There  were  46  system 
capabilities  assessed  in  this  version  of  the  DSTS  Performance  Capabilities  Checklist.  The 
specific  capabilities  were  relevant  to  the  purpose  of  Bold  Quest  and  to  the  scenarios  used.  All 
other  aspects  of  the  DSTS  System  Capabilities  Checklist  were  similar  to  the  content  and  format 
of  the  DSTS  Manned  Module  Capabilities  Checklist.  The  DSTS  System  Capability  Checklist 
(with  response  data)  is  presented  in  Appendix  H. 

After  Action  Review  Capabilities  Questionnaire.  The  After  Action  Review  Capabilities 
Questionnaire  was  used  to  get  input  from  Squad  members  on  the  usefulness  of  the  AAR 
capabilities  of  DSTS.  The  questionnaire  contained  four  items  about  playback  and  viewpoint 
capabilities.  Each  item  was  rated  on  “effectiveness”  using  a  4-point  scale.  The  After  Action 
Review  Capabilities  Questionnaire  is  presented  in  Appendix  I. 

Decision  Making  Questionnaire.  The  Decision  Making  Questionnaire  was  used  to 
determine  if  the  scenario-based  training  in  either  the  live  environment  or  virtual  environment 
(i.e.,  DSTS)  offered  a  reasonable  opportunity  for  Soldiers  to  practice  tactical  decision-making 
skills.  The  questionnaire  contained  10  items  that  focused  on  aspects  of  the  training  that 
potentially  could  challenge  decision  making  and  on  a  Soldiers’  decision-making  confidence  as  a 
result  of  the  training.  Each  item  was  rated  on  a  4-point  scale  as  appropriate  to  each  query.  The 
Decision  Making  Questionnaire  is  presented  in  Appendix  J. 

Training  Preparation  Questionnaire.  The  Training  Preparation  Questionnaire  was  used 
to  assess  Soldiers’  perspectives  on  the  degree  to  which  the  training  environment  prepared  them 
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for  final  mission  execution.  There  were  six  items  on  the  questionnaire,  and  each  item  was  rated 
on  a  4-point  scale  as  appropriate  for  the  items.  A  version  of  this  questionnaire  was  worded 
specifically  for  the  live-training  environment,  and  a  version  was  worded  for  the  virtual-training 
environment.  Both  versions  of  the  questionnaire  are  presented  in  Appendix  K. 

Operational  Realism  Questionnaire.  The  Operational  Realism  Questionnaire  was 
designed  to  assess  Soldiers’  perspectives  on  the  functional  verisimilitude  of  the  training 
environment  as  related  to  mission  execution.  That  is,  the  items  on  this  questionnaire  were 
designed  to  assess  whether  or  not  the  physical  cues  were  present  in  the  training  environment  to 
effectively  execute  a  given  mission.  The  questionnaire  contained  1 1  items.  Each  item  was  rated 
for  “sufficiency”  on  a  4-pomt  scale.  The  Operational  Realism  Questionnaire  is  presented  in 
Appendix  L. 

Procedure.  The  Experimental  Eorce  squads  were  assigned  to  either  the  live-training 
condition  or  the  simulation-training  condition  (i.e.,  DSTS).  This  assignment  was  determined  by 
the  Experimental  Eorce  leadership.  Both  squads  followed  the  same  general  sequence  of  events. 
Each  squad  first  executed  a  live  area  reconnaissance  mission,  spent  three  days  training  in  their 
respective  training  environments,  and  again  executed  a  live  area  reconnaissance  mission. 
Because  of  the  experimental  demands  on  the  Soldiers  during  Bold  Quest,  the  sequence  of  events 
was  spread  across  two  weeks.  Both  squads  executed  their  initial  live  mission  execution  on 
Tuesday  of  the  first  week.  The  squads  then  began  the  three-day  training  on  Monday  of  the 
following  week  and  executed  the  final  live  mission  on  Thursday  of  that  week.  Each  day  of 
training  consisted  of  7  hours  of  training  time. 

The  live  mission  execution  and  live  training  were  conducted  at  the  McKenna  Military 
Operations  in  Urban  Terrain  (MOUT)  training  facility  at  Eort  Benning,  GA.  Eive  role  players 
were  used  for  the  opposing  force  and  for  non-combatants  on  the  battlefield.  All  Soldiers  in  live 
mission  execution  and  live  training  were  instrumented  with  Multiple  Integrated  Easer 
Engagement  System  equipment.  During  live  mission  execution,  the  area  reconnaissance 
scenario  was  augmented  with  an  inject  that  required  the  squad  to  notice  hostile  activity  and  to 
take  appropriate  offensive  action.  Each  squad  participated  in  an  AAR  after  the  initial  mission 
execution  and  again  after  the  final  mission  execution.  The  purpose  of  the  AAR  was  to  obtain  a 
squad  self-assessment  of  performance.  The  AAR  was  lead  by  a  combat-experienced  member  of 
the  Bold  Quest  analysis  team. 

The  live-training  squad  was  allowed  to  execute  any  desired  training  during  the  three-day 
training  period.  The  squad  used  the  first  day  to  practice  battle  drills  and  standard  operating 
procedures  for  area  reconnaissance.  On  the  second  and  third  days,  the  squad  executed  mission 
rehearsals  with  increasing  degrees  of  complexity  and  opposing  force  interaction.  The  live- 
training  squad  did  utilize  feedback  from  MOUT  and  opposing  force  observers-controllers.  In 
addition,  the  squad  conducted  AARs  after  each  mission  rehearsal. 

As  with  the  live-training  squad,  the  DSTS  squad  was  allowed  to  execute  any  desired 
training  over  the  three-day  training  period.  Before  training  started,  the  squad  spent  about  half  of 
a  day  with  familiarization  instruction  from  the  DSTS  operators,  practice  donning  the  VSMM, 
and  execution  of  a  calibration  and  familiarization  scenario.  The  squad  used  the  remainder  of  the 
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first  day  of  training  and  most  of  the  second  day  of  training  executing  mission-rehearsal 
scenarios.  The  remainder  of  the  training  time  was  divided  between  executing  battle  drills  (e.g., 
room  clearing  drills)  and  mission  rehearsal. 

Each  squad  executed  the  final  live  area  reconnaissance  mission  at  the  MOUT  facility  on 
the  same  day.  The  procedure  was  the  same  for  each  squad.  The  final  mission  started  with  a 
mission  brief  from  the  Platoon  Leader,  the  squad  then  assembled  in  a  staging  area  for  pre-combat 
checks  and  pre-combat  inspections.  When  given  the  signal  from  exercise  controllers,  the  squad 
moved  to  a  rally  point  and  began  ingress  on  the  objective.  From  here,  the  mission  scenario 
guided  action,  but  the  specific  events  depended  on  the  decisions  and  actions  made  by  the  squad. 
After  the  mission  was  competed,  the  squad  participated  in  an  AAR.  Finally,  the  squad 
completed  a  set  of  questionnaires.  All  questionnaires  were  administered  using  a  laptop 
computer.  Each  of  the  questionnaires  listed  in  Appendices  I  through  M  were  formatted  for  the 
computer  and  presented  one  item  at  a  time.  The  live-training  squad  was  the  first  to  execute  the 
final  live  mission  and  completed  the  entire  sequence  of  events  before  the  DSTS  squad  began. 

The  live-training  squad  only  completed  the  Decision  Making  Questionnaire,  the  Training 
Preparation  Questionnaire,  and  the  Operational  Realism  Questionnaire,  and  the  simulation¬ 
training  squad  completed  all  five  questionnaires  (i.e.,  the  same  three  as  the  live-training  squad 
plus  the  DSTS  Performance  Capabilities  Checklist  and  the  After  Action  Review  Capabilities 
Questionnaire). 

Results 

DSTS  Performance  Capabilities  Checklist  The  responses  to  the  DSTS  Performance 
Capabilities  Checklist  were  aggregated  across  respondents,  and,  again,  each  capability  was 
classified  as  either  “Green,”  “Amber,”  or  “Red”  by  similar  convention  as  the  User  Assessment 
data.  With  fewer  participants  in  Bold  Quest  than  the  User  Assessment  (i.e.,  9  vice  18)  and  with 
only  a  single  assessment  event  for  Bold  Quest  (vice  daily  assessments  in  the  User  Assessment), 
there  was  much  less  data  available  from  Bold  Quest  for  analysis  as  compared  to  the  User 
Assessment.  In  addition,  there  was  considerable  agreement  among  participants  on  responses  to 
performance  capabilities,  and,  as  a  consequence,  there  were  empty  response  cells  (e.g.,  “Exactly 
Like”  for  Similarity  ratings)  for  each  performance  capability.  Because  of  the  nature  of  the  data, 
statistical  testing  (i.e.,  chi-square  test)  was  not  possible,  but  inferences  were  easily  drawn.  Each 
capability  was  classified  “Green,”  “Amber,”  or  “Red”  based  on  the  majority  of  responses.  There 
were  few  cases  where  the  classification  decision  was  ambiguous,  but  when  ties  existed,  the 
classification  was  assigned  to  the  “Amber”  category. 

Twelve  of  the  46  performance  capabilities  assessed  were  either  not  included  in  the  Area 
Reconnaissance  scenario,  not  utilized  during  training,  or  could  not  be  performed  by  Soldiers. 
Those  12  capabilities  are  listed  in  bold  font  in  Appendix  H.  Of  the  remaining  34  capabilities,  the 
vast  majority  was  classified  as  “Red”  for  “Similar”  ratings  (24),  for  “Accurately”  ratings  (26), 
and  for  “Difficult”  ratings  (23).  Many  fewer  performance  capabilities  were  classified  as 
“Amber”  for  “Similar”  ratings  (9),  for  “Accurately”  ratings  (6),  and  for  “Difficult”  ratings  (8), 
and  almost  no  capabilities  were  classified  as  “Green”  for  “Similar”  ratings  (1),  for  “Accurately” 
ratings  (1),  and  for  “Difficult”  ratings  (3).  Table  4  lists  the  performance  capabilities  classified  as 
“Amber”  and  “Green”  for  each  rating.  A  quick  review  of  Table  4  indicates  that  the  performance 


24 


capabilities  perceived  as  sufficient  for  training  (i.e.,  “Amber”  or  “Green”)  were  based  on  postural 
skills  (e.g.,  standing  and  aiming  a  weapon)  or  visual  identification  (e.g.,  identify  own 
Squad/team  members  and  identify  objects). 

Table  4 

DSTS  Performance  Capabilities  Classified  as  Green  and  Amber  in  Bold  Quest. 


Performance  Capability 

Similar 

Ratings 

Accurately 

Difficult 

Identified  terrain  features 

G 

Stand,  Kneel,  Prone 

G 

G 

A 

Look 

A 

Move  up  and  down  stairs 

A 

A 

A 

Maintain  position  in  formation 

A 

A 

A 

Maintain  balance  while  moving 

A 

A 

A 

Carry  casualties 

G 

Drag  casualties 

G 

Identify  own  squad/fire  team  members 

A 

A 

Know  location  of  Squad/team  members 

A 

Aim  weapon  (apply  lead  or  hold-off) 

A 

Fire  weapon 

A 

A 

A 

Reload  weapon 

A 

A 

A 

Engage  targets  from  standing 

A 

Identify  objects 

A 

Note:  G  =  Green  classification;  A  =  Amber  classification 


Unfortunately,  the  Accurately  ratings  did  not  add  discrimination  among  the  ratings  of 
performance  capabilities.  In  fact,  it  appeared  that  most  performance  capabilities  were  perceived 
as  either  “useful”  or  “not  useful”  and  those  perceptions  resulted  in  consistent  responses  across 
ratings.  Unlike  the  system-capability  ratings  for  the  User  Assessment,  the  ratings  for  Bold  Quest 
presented  a  generally  negative  impression  of  the  potential  training  effectiveness  of  DSTS.  It  is 
more  likely  that  the  negative  perception  of  DSTS  by  Bold  Quest  participants  was  due  to  the  lack 
of  familiarity  with  the  system  rather  than  true  insufficiencies  in  the  system.  The  Bold  Quest 
participants  had  much  less  time  to  use  the  system  than  did  User  Assessment  participants. 
Likewise,  the  Bold  Quest  participants  only  used  DSTS  to  train  and  execute  one  type  of  mission 
(i.e.,  area  reconnaissance),  whereas  User  Assessments  participants  executed  several  types  of 
missions.  Also,  the  proportion  of  “Red”  responses  decreased  in  the  User  Assessment  as  the 
amount  of  time  using  the  system  increased.  Thus,  it  appeared  that  the  discrepancy  in  Soldiers’ 
perspectives  of  DSTS  across  the  two  experiments  emphasizes  the  high  level  of  system  familiarity 
required  to  obtain  training  benefit  from  DSTS. 


25 


After  Action  Review  Capabilities  Questionnaire.  The  responses  to  the  items  on  the 
After  Action  Review  Capabilities  Questionnaire  (see  Appendix  I)  were  first  compared  across 
items  to  determine  the  relative  usefulness  of  the  capabilities.  There  was  no  statistical  difference 
in  the  patterns  of  responses  across  items  (x  (9)  =  1 .78).  The  responses  were  then  aggregated 
across  items  to  determine  the  overall  perception  of  the  AAR  capabilities.  The  analysis 
aggregated  the  responses  into  two  categories:  Not  effective  (i.e.,  “not  effective”  and  “somewhat 
effective”  responses);  and  Effective  (i.e.,  “mostly  effective”  and  “very  effective”  responses). 

This  analysis  indicated  that  Soldiers  perceived  the  AAR  capabilities  as  effective  (x^(l)  =  5.76). 

Decision  Making  Questionnaire.  The  response  frequencies  for  each  item  of  the 
Decision  Making  Questionnaire  (see  Appendix  J)  were  compared  across  the  training 
environments  (i.e.,  DSTS  vs.  live  training).  None  of  the  items  produced  a  statistical  difference 
between  groups  (highest  x  (3)  =  6.08).  Thus,  it  appeared  that  DSTS  provided  opportunities  and 
capabilities  equal  to  but  no  better  than  live  training.  Even  though  each  item  on  the  Decision 
Making  Questionnaire  was  based  on  a  different  dimension  (e.g.,  confident  vs.  improvement),  it 
was  clear  that  the  majority  of  responses  indicated  that  neither  DSTS  nor  live  training  provided 
much  benefit  to  decision-making  ability. 

Training  Preparation  Questionnaire.  Once  again,  response  frequencies  for  each  item 
on  the  Training  Preparation  Questionnaire  (see  Appendix  K)  were  compared  across  the  training 
environments  (i.e.,  DSTS  and  live  training).  Group  differences  were  found  for  five  of  the  six 
items.  The  pattern  of  group  differences  can  be  summarized  by  saying  the  Soldiers  in  live 
training  perceived  themselves  as  more  prepared  to  execute  the  final  live  mission  than  were 
Soldiers  training  in  DSTS.  Eigure  6  shows  responses  combined  into  “effective”  responses  (e.g., 
“moderate  improvement”  and  “significant  improvement”)  and  “not  effective”  responses  (e.g., 

“no  improvement”  and  “some  improvement”)  for  each  Training  Preparation  Questionnaire  item 
across  the  training  environments.  As  can  be  seen,  the  “effective”  responses  for  live  training  were 
statistically  greater  than  for  DSTS  (x^(3)  >  9.90).  The  one  exception  to  this  pattern  was  for  the 
item  that  addressed  the  awareness  of  what  to  expect  from  the  live  mission  (x  (3)  =  6.37).  While 
the  Soldiers  in  live  training  had  more  “effective”  responses  than  Soldiers  using  DSTS  for  this 
item,  the  majority  of  responses  for  Soldiers  in  DSTS  were  also  “effective.” 
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DSTS  Live  DSTS  Live  DSTS  Live  DSTS  Live  DSTS  Live  DSTS  Live 


Impioved  Executed  Aware  of  Improved  Made  you  Increased 

your  more  qurcklyi  your  role  mrssron  prepared  |  expectations 

perfoimance  success  of  mission 

/^(3)=16.55  /^(3)=10.99  ;^^(3)=13.78  /^(3)  =  11.:7  /^(3)  =  9.91  /^(3)  =  6.37 

Figure  6.  Percent  of  Responses  on  the  Training  Preparation  Questionnaire  between  Live 
Training  and  Dismounted  Soldier  Training  System  (DSTS). 

Operational  Realism  Questionnaire.  The  response  frequencies  were  aggregated  aeross 
Operational  Realism  Questionnaire  items  (see  Appendix  L)  for  eaeh  training  environment  (i.e., 
DSTS  and  live  training)  to  yield  overall  ratings  of  operational  realism.  The  aggregated 
frequeneies  were  then  eompared  aeross  the  two  training  environments.  Overall,  the  majority  of 
responses  for  eaeh  training  environment  were  in  the  “generally  suffieient”  eategory.  However, 
Soldiers  in  live  training  had  more  responses  in  the  “generally  suffieient”  and  “very  suffieient” 
eategories  and  fewer  responses  in  the  “generally  insufficient”  and  “very  insuffieient”  eategories 
as  eompared  to  Soldiers  in  DSTS  (x^(3)  =  145.28).  This  pattern  of  responses  is  presented  in 
Figure  7,  whieh  provides  the  pereent  of  responses  for  eaeh  training  environment  aeross  the 
response  eategories.  Not  surprisingly,  the  results  indicated  that  live  training  provided  more 
pereeived  funetional  verisimilitude  than  DSTS.  Soldiers  who  trained  in  DSTS  provided 
statistieally  equivalent  “suffieient”  responses  and  “insuffieient”  responses  (x  <  1).  As  a 
eonsequenee,  it  was  difficult  to  determine  if  DSTS  was  pereeived  as  providing  suffieient 
funetional  verisimilitude  for  training  infantry  tasks. 
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Figure  7.  Percent  of  Responses  on  the  Operational  Realism  Questionnaire  aeross  Live  Training 
and  Dismounted  Simulation  Training  System  (DSTS). 


Summary  of  Bold  Quest  Data.  Bold  Quest  data  provided  both  an  absolute  view  and  a 
relative  view  of  the  training  effeetiveness  of  DSTS.  Soldiers  training  with  DSTS  provided  their 
pereeptions  of  the  eapabilities  of  DSTS.  Some  of  those  pereeptions  were  eontrasted  with 
pereeptions  of  Soldiers  in  live  training  with  the  same  training  objeetives.  Responses  from 
Soldiers  training  with  DSTS  overwhelmingly  indieated  that  DSTS  was  not  effective.  That  is,  the 
majority  of  performanee  eapabilities  were  rated  as  “Red,”  high  frequeneies  of  non-affirming 
responses  given  on  the  Deeision  Making  Questionnaire,  low  frequeneies  of  “effeetive”  responses 
given  on  the  Training  Preparation  Questionnaire,  and  equivalent  frequeneies  of  “effieient” 
responses  and  “insufficient”  responses  on  the  Operational  Realism  Questionnaire.  However,  the 
AAR  capabilities  of  DSTS  were  perceived  as  being  effective. 


More  importantly,  responses  from  Soldiers  training  in  DSTS  differed  from  responses 
from  Soldiers  training  in  the  live  environment.  Soldiers  in  live  training  provided  more 
“effeetive”  responses  than  did  Soldiers  using  DSTS  on  the  Training  Preparation  Questionnaire, 
and  Soldiers  in  live  training  generally  rated  the  training  as  “effeetive”  in  preparing  them  for  final 
mission  exeeution.  Likewise,  Soldiers  in  live  training  provided  more  “suffieient”  responses  on 
the  Operational  Realism  Questionnaire  compared  to  Soldiers  using  DSTS  and  mostly  rated  the 
live  training  as  “sufficient.”  These  differences  indicated  that  DSTS  was  perceived  as  less 
effeetive  than  live  training.  Whereas  live  training  was  pereeived  as  “effeetive”  and  “suffieient,” 
DSTS  was  perceived  as  “not  effeetive”  and  “insufficient.” 
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Discussion 


The  data  from  the  DSTS  User  Assessment  and  from  Joint  Forees  Bold  Quest  2012 
doeumented  the  realism  of  Soldiers’  abilities  to  move,  shoot,  eommunieate,  and  navigate  in  the 
virtual  environment  as  well  as  other  indieators  of  training  effeetiveness.  The  results  eould  be 
elassified  as  “mixed”  at  best.  Performanee  eapabilities  were  mostly  classified  as  acceptable  (i.e.. 
Amber  or  Green)  in  the  User  Assessment,  but  the  majority  of  performance  capabilities  were 
rated  as  unacceptable  in  Bold  Quest  (i.e..  Red).  Likewise,  Soldiers  in  the  User  Assessment 
reported  feeling  immersed  in  DSTS,  but  Soldiers  in  Bold  Quest  reported  a  lower  sense  of 
operational  realism  in  DSTS  compared  to  live  training.  Not  only  was  the  sense  of  operational 
realism  lower  in  DSTS  than  live  training,  but  also  the  sense  of  training  preparation  was  lower  in 
DSTS  than  live  training  during  Bold  Quest.  However,  the  DSTS  AAR  capabilities  were 
universally  perceived  as  providing  a  positive  impact  on  training. 

Some  of  the  inconsistencies  seen  in  the  data  may  be  attributed  to  the  amount  of  DSTS 
exposure  across  the  two  capabilities  experiments.  The  Bold  Quest  participants  had  much  less 
time  to  use  the  DSTS  than  did  User  Assessment  participants.  Moreover,  Bold  Quest  participants 
only  used  DSTS  to  train  and  execute  one  type  of  mission  (i.e.,  area  reconnaissance),  whereas 
User  Assessment  participants  executed  several  types  of  missions.  As  was  seen  in  the  User 
Assessment  data,  more  than  8  hours  of  structured  familiarization  training  were  required  before 
Soldiers  were  comfortable  enough  with  DSTS  to  effectively  execute  training.  It  was  also 
observed  that  DSTS  training  during  Bold  Quest  focused  on  executing  buddy-team  movement  and 
room  clearing.  These  tasks  are  effectively  trained  in  DSTS  but  had  little  to  do  with  the  mission 
execution  on  which  DSTS  was  assessed  (i.e.,  area  reconnaissance).  As  a  consequence.  Soldiers 
in  Bold  Quest  may  have  perceived  little  benefit  of  DSTS  training  on  performance  of  final 
mission  execution. 

Overall,  the  capabilities  experiments  provided  opportunities  to  observe  and  assess  the 
new  generation  of  Dismounted  Infantry  simulation  technology  and  capabilities.  Dismounted 
Infantry  simulator  capabilities  seem  to  have  significantly  increased  from  VSTS  in  2007  to  DSTS 
in  2012.  Game  engines  have  increased  in  fidelity  and  ability  to  replicate  real-world  settings, 
equipment,  battlefield  effects,  and  friendly  and  enemy  demographics.  Replica  weapons  closely 
match  form,  fit,  and  functionality  of  real  weapons.  Soldiers  are  now  completely  un-tethered  and 
can  control  their  avatars’  movements  through  natural  locomotion.  Simulator  flexibility  and 
scenario  development  capability  can  expose  Soldiers  to  ever-increasing  complex  situations  while 
conducting  collective  task  training  that  is  not  currently  available  in  home-station  live  training. 
AAR  systems  provide  a  tool  for  leaders  to  view  the  training  exercise  from  any  vantage  point  and 
provide  cause-and-effect  feedback  to  improve  proficiency  and  to  mitigate  training  gaps. 

However,  the  training-capability  data  reported  here  revealed  that  there  are  still  some 
shortfalls  in  simulator  capabilities  that  hinder  the  ability  to  completely  immerse  the  Soldiers  in 
the  virtual  environment  and  to  eliminate  distractors  to  training.  These  shortfalls  are  not  new  to 
the  current  generation  of  simulators  but  are  repetitive  in  nature.  Clearly,  more  is  needed  for  both 
the  development  of  Dismounted  Infantry  simulation  and  for  the  evaluation  of  Dismounted 
Infantry  simulation  training  effectiveness.  Additional  analyses  of  the  current  data  could  help 
identify  specific  DSTS  training  capabilities  that  support  successful  skills  transfer  or  inform  the 
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progress  of  Dismounted  Infantry  simulation  as  compared  to  historical  trends  identified  in 
previous  research.  Additional  data  could  be  used  to  discover  ways  to  increase  DSTS  utilization 
and  effectiveness  or  to  identify  the  relative  advantages  of  other  types  of  Dismounted  Infantry 
simulation  technologies  (e.g.,  VIRTSIM). 

The  data  documented  in  this  report  were  used  to  inform  decisions  from  the  DSTS  User 
Assessment.  Some  of  the  data  were  also  included  in  the  Army  Expeditionary  Warrior 
Experiment  -  Bold  Quest  2012  report  (U.S.  Army  Evaluation  Center,  2013).  In  addition,  the 
results  were  briefed  to  Program  Manger  -  Combined  Arms  Tactical  Trainer,  Training  and 
Doctrine  Command  Capabilities  Manager  -  Virtual  Training,  and  Maneuver  Center  of 
Excellence  Director  of  Training  and  Doctrine. 
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APPENDIX  A 

SOLDIER  BACKGROUND  QUESTIONNAIRE 


A-1 


Soldier  Background  Questionnaire 


1 .  What  is  your  age? _ years. 

2.  Rank _ 

3. MOS  (e.g.,  IIB,  IIC) _ 

4.  Time  in  service;  Years _  Months _ 

5.  Primary  Infantry  experience:  (Check  one) 

Light _  Wheeled  (Stryker) _  Mechanized 

6.  Current  unit  (Squad,  Platoon,  and  Company)? _ 

7.  How  long  have  you  been  in  this  unit?  Years _  Months _ 

8.  What  is  your  current  duty  position? 

_ Platoon  Leader/Platoon  Sergeant 

_ Squad  leader 

_ Fire  team  leader 

_ Rifleman 

_ M203/320  gunner 

_ SAW/M249  gunner 

Other _ 

9.  How  long  have  you  held  this  position? _ 

10.  Which  of  the  following  positions  have  you  held?  For  how  long? 


Check  all  that  apply 

Unit 

Special  Assignment 

How  long? 

How  long? 

Team  Leader 

Drill  Sergeant 

Squad  Leader 

Recruiter 

Section  Leader 

Other: 

Platoon  LDR/SGT 

1 1 .  Number  of  combat  deployments? 
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12.  If  deployed,  what  duty  positions  did  you  hold  when  deployed?  List  from  most  frequent  to 
least  frequent.  If  not  deployed,  skip  to  item  14. 


13.  If  deployed,  list  the  three  types  of  missions  you  most  often  executed  in  theater. 


14.  What  optics/sights  have  you  used  on  your  weapon? 


15.  What  night  equipment  have  you  used? 


16.  Are  you  color  blind?  _ Yes _ No 

17.  Are  you _ right  handed?  _ left  handed? 
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18.  How  susceptible  to  motion  or  oar  sickness  do  you  feel  you  are?  (Check  one.) 


0 

1 

2 

3 

4 

5 

6 

7 

Not 

Susceptible 

Very 

Mildly 

Average 

Very 

Highly 

19.  What  is  you  level  of  oonfidenoe  in  using  computers?  (Check  one) 


1 

2 

3 

4 

5 

Low 

Average 

High 

20.  How  many  hours  per  week  do  you  use  oomputers? _ 

21 .  How  many  hours  per  week  do  you  play  computer  games? _ 

22.  What  are  your  primary  uses  of  a  oomputer?  (Cheek  all  that  apply) 

_ Sending/replying  to  e-mail 

_ Searohing  the  internet 

_ Entering  data  /reoords  for  your  unit 

_ Writing  documents 

_ Developing  Power  Point  slides 

_ Gaming 

_ Web  surfing  /  Entertainment 

_ Other: _ 

23.  Have  you  had  any  other  experienee  with  military  computer  simulations  such  as  the  Close 
Combat  Taetieal  Trainer  (CCTT)  or  a  dismounted  Infantry  simulator? 

_ Yes _ No 

If  yes,  please  give  the  names  of  the  simulators  or  briefly  describe  them. 


24.  How  useful  do  you  think  simulation  is  for  training  squad-level  tactical  tasks?  (Check  one) 
Not  Somewhat  ^  ,  Extremely 
Useful  Useful  Useful 


25.  How  much  of  live  squad-level  training  CAN  simulation  training  replaee?  (Cheek  one) 


None  Some  Most  All 
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26.  How  much  of  live  squad-level  training  SHOULD  simulation  training  replace?  (Check  one) 


None  Some  Most  All 


27.  Do  you  look  forward  to  the  ability  to  train  squad-level  task  with  simulation?  (Check  one) 


Not  at 
All 


Undecided 


Mostly  Definitely 


28.  What  do  you  see  as  the  advantages  of  using  simulation  training  for  squad-level  task? 
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APPENDIX  B 

SQUAD  TACTICAL  PROFICIENCY  ASSESSMENT  - 
PLATOON  LEADER/PLATOON  SERGEANT 
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Frequency 

Recency 

Environment 

Evaluation 

Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

Less 
than  1 
month 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

Shoot 

Call  for  and 

Adjust  Indirect 
Fire 

3 

0 

2 

2 

I 

I 

2 

I 

2 

I 

2 

I 

0 

Engage  Targets 
with  organic 
indirect  fire 
systems  (M203, 
M320) 

(day/night) 

I 

I 

I 

0 

0 

I 

0 

0 

0 

I 

I 

0 

0 

Move 

Control 

Movement  of  a 
Fire  Team 

3 

0 

0 

2 

I 

I 

2 

0 

3 

0 

3 

0 

0 

Move  Under 
Direct  Fire 

2 

I 

I 

I 

0 

0 

2 

0 

2 

0 

2 

0 

0 

React  to  Indirect 
Fire  While 
Dismounted 

2 

I 

0 

0 

0 

0 

2 

0 

2 

0 

2 

0 

0 

B-2 


Observed 

Not  Observed 

Frequency 

Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

Less 
than  1 
month 

Move  Over, 
Through,  or 
Around 

Obstacles 

(Except 

Minefields) 

1 

2 

0 

1 

0 

0 

Select  an 

Overwatch 

Position 

1 

2 

1 

0 

0 

0 

Conduct 
Movement 
Techniques  by  a 
Squad 

3 

0 

1 

1 

1 

1 

Conduct  the 
Maneuver  of  a 
Squad 

3 

0 

1 

2 

0 

1 

Enter  a 

Building/Clear  a 
Room  During  an 
Urban  Operation 

3 

0 

1 

2 

0 

1 

Use  a  Map 
Overlay 

3 

0 

2 

1 

0 

1 

Analyze  Terrain 

3 

0 

1 

2 

0 

1 

Communicate 


Recency 

Environment 

Evaluation 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

1 

0 

1 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 

2 

0 

3 

0 

2 

1 

0 

2 

0 

3 

0 

2 

1 

0 

2 

0 

3 

0 

2 

1 

0 

2 

0 

2 

0 

2 

1 

0 

2 

0 

2 

0 

2 

1 

0 

B-3 


Observed 

Not  Observed 

Frequency 

Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

Less 
than  1 
month 

Report 

Casualties 

3 

0 

0 

1 

2 

1 

Perform  Voice 
Communications 

3 

0 

1 

2 

0 

1 

Request  Medical 
Evacuation 

3 

0 

1 

1 

1 

1 

Use  Visual 

Signaling 

Techniques 

1 

2 

0 

2 

0 

0 

Issue  a 

Fragmentary 

Order 

2 

1 

0 

1 

0 

0 

Issue  a  Warning 
Order 

1 

2 

0 

2 

0 

0 

Issue  an  Oral 
Operation  Order 

1 

2 

0 

1 

0 

0 

Plan  for  Use  of 
Supporting  Fires 

1 

2 

0 

0 

0 

0 

Offensive  Collective  Tasks 


Recency 

Environment 

Evaluation 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

2 

0 

3 

0 

3 

0 

0 

2 

0 

3 

0 

3 

0 

0 

2 

0 

3 

0 

3 

0 

0 

2 

0 

2 

0 

0 

2 

0 

2 

0 

2 

0 

0 

2 

0 

2 

0 

2 

0 

0 

2 

0 

1 

0 

1 

0 

0 

2 

0 

1 

0 

1 

0 

0 

0 

0 

B-4 


Observed 

Not  Observed 

Frequency 

Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

Less 
than  1 
month 

Conduct  an 

Attack  by  a 
Platoon 

2 

1 

1 

1 

0 

0 

Conduct  a  Raid 

2 

1 

1 

1 

0 

0 

Conduct 
Occupation  of 
an  Overwatch 
Position 

1 

2 

0 

1 

0 

0 

Conduct  a 

Breach  of  a 

Wire  Obstacle 

1 

2 

0 

1 

0 

0 

Conduct  an 

Attack  on  a 
Building  by  a 
Squad  During  an 
Urban  Operation 

2 

1 

0 

2 

0 

0 

Consolidate  a 

Unit 

2 

1 

1 

1 

0 

0 

Reorganize  a 

Unit 

2 

1 

0 

1 

1 

0 

Defensive  Collective  Tasks 


Recency 

Environment 

Evaluation 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

1 

1 

2 

0 

2 

0 

0 

1 

1 

2 

0 

2 

0 

0 

1 

0 

1 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 

2 

0 

2 

0 

2 

0 

0 

2 

0 

2 

0 

2 

0 

0 

2 

0 

2 

0 

2 

0 

0 
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Frequency 


Conduct  a 
Defense  by  a 
Squad 


Conduct  a 
Defense  by  a 
Platoon 


Conduct 
Occupation  of 
an  Assembly 
Area 


Conduct  a 

Leader's 

Reconnaissance 


Establish  an 
Observation 
Post 


Supervise 
Construction  of 
a  Fighting 
Position 


Coordinate  with 
an  Adjacent 
Platoon 


Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

0 

1 

0 

1 

1 

0 

1 

1 

0 

1 

1 

0 

1 

0 

0 

1 

0 

0 

0 

2 

0 

Less 
than  1 
month 


Recency 

Environment 

Evaluation 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

1 

0 

1 

0 

1 

0 

0 

2 

0 

2 

0 

1 

1 

0 

2 

0 

2 

0 

1 

1 

0 

2 

0 

2 

0 

1 

1 

0 

1 

0 

1 

0 

1 

0 

0 

1 

0 

1 

0 

0 

1 

0 

2 

0 

2 

0 

1 

1 

0 

B-6 


Frequency 


O 


O 

O 


Once  or 
Twice 


3  to  5 
Times 


More 
than  5 
times 


Less 
than  1 
month 


Control  Organic 
Fires 


Conduct  a 
Defense  by  a 
Platoon  During 
an  Urban 
Operation 


Select  Hasty 
Firing  Positions 
During  an  Urban 
Operation 


Prepare 
Positions  for 
Individual  and 
Crew- Served 
Weapons  During 
an  Urban 
Operation 


Recency 

Environment 

Evaluation 

1  to  6 
months 

6  to  12 
months 

Live 

(dry, 

blank, 

live- 

fire) 

Virtual 

Trained 

Practiced 

Untrained 

2 

0 

2 

0 

1 

1 

0 

1 

0 

1 

0 

1 

0 

0 

2 

0 

2 

0 

2 

0 

0 

1 

0 

2 

0 

1 

1 

0 
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APPENDIX  C 

SQUAD  TACTICAL  PROFICIENCY  ASSESSMENT  -  SQUAD  LEADERS 
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Frequency 


Engage  Targets 
During  an  Urban 
Operation 


0 


0 


Recency 

Environment 

Evaluation 

Less 
than  1 
month 

1  to  6 
months 

6  to  12 
months 

Live 
(dry, 
blank, 
live -fire) 

Virtual 

Trained 

Practiced 

Untrained 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

2 

0 

2 

0 

0 

0 

0 

1 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 

C-2 


Control 

Movement  of  a 
Fire  Team 


0 


0 


Recency 

Environment 

Evaluation 

Less 
than  1 
month 

1  to  6 
months 

6  to  12 
months 

Live 
(dry, 
blank, 
live -fire) 

Virtual 

Trained 

Practiced 

Untrained 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

C-3 


-o 

u 

> 


Select  an 

Overwatch 

Position 

Conduct 
Movement 
Techniques  by  a 
Squad 


0 


1 


Perform 

Movement 

Techniques  1 

During  an  Urban 
Operation 


Enter  a  Building 
During  an  Urban  1 
Operation 


Use  a  Map  ^ 

Overlay 

Analyze  Terrain  1 


O 


2 


1 


1 


1 


1 


1 


Select  a 
Movement 
Route  Using  a 
Map 


o 


Once  or 
Twice 


0 


0 


0 


0 


0 


0 


0 


2 


0 


C-4 


Select 

Temporary 

Fighting 

Positions 

Construct 

Individual 

Fighting 

Positions 

Clear  a  Field  of 
Fire 

Transport  a 
Casualty 

Communicate 

Report 

Casualties 

Perform  Radio 
Voice 

Communications 

Request  Medical 
Evacuation 

Use  Visual 

Signaling 

Techniques 


Recency 

Environment 

Evaluation 

Less 
than  1 
month 

1  to  6 
months 

6  to  12 
months 

Live 
(dry, 
blank, 
live -fire) 

Virtual 

Trained 

Practiced 

Untrained 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 
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Observed 

Not  Observed 

Frequency 

Recency 

Environment 

Evaluation 

Once  or 
Twice 

3  to  5 
Times 

More 
than  5 
times 

Less 
than  1 
month 

1  to  6 
months 

6  to  12 
months 

Live 
(dry, 
blank, 
live -fire) 

Virtual 

Trained 

Practiced 

Untrained 

Offensive  Collective  Tasks 

Conduct  a 

Breach  of  a 

Wire  Obstacle 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Defensive  Collective  Tasks 

Establish  an 
Observation 

Post 

0 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Supervise 
Construction  of 
a  Fighting 

Position 

1 

1 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 
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APPENDIX  D 

DSTS  MANNED  MODULE  OFFENSE  CAPABILITES  CHECKLIST 


D-1 


DSTS  Manned  Module  Daily  Checklist 

This  checklist  will  be  used  to  document  the  DSTS  Manned  Module  capabilities.  The  results  of  this  information  will  be  used 
to  assess  the  Dismounted  Soldier  Training  System.  The  primary  user  of  this  document  is  the  research  team  member. 
Follow  the  steps  below  to  complete  this  checklist. 

❖  Observe  scenario  execution  from  assigned  position,  i.e.  SL  Observer,  A  Tm  Observer,  B  Tm 
Observer, 

❖  Annotate  either  "SL",  "A  Tm",  or  "B  Tm"  on  ID  iine,  date  on  Date  iine,  type  of  scenario  on  Scenario 
iine,  and  start  and  stop  times 

❖  Annotate  for  each  task  whether  you  observed  or  did  not  observe  the  task  being  accompiished  or 
executed,  not  aii  tasks  wiii  be  executed  during  each  scenario, 

>  E.g.  A  Team  used  map  to  identify  terrain  features  (see  exampie  tabie  beiow), 

❖  At  the  end  of  the  day  interview  Squad  Leader  or  Fire  Team  as  assigned, 

❖  Provide  copy  of  tabie,  1  per  2  Soidiers, 

❖  If  task  not  observed,  ask  Soldiers  if  they  “Performed”,  “Did  not  perform”,  or  “Could  not  perform”,  mark 
as  appropriate, 

❖  Ask  questions,  “How  Similar”,  “How  Quickly”,  “How  Difficult”,  based  on  Task  and  subtasks,  i.e. 
Navigate  -  conducted  map  recon, 

❖  Annotate  number  of  responses  in  each  box, 

>  E.g.  there  are  4  team  members  in  A  Team  -  2  Team  members  stated  Id'ing  terrain  features  was 
very  similar  to  the  real  world,  while  1  stated  very  different  and  1  stated  exactly  alike  (see 
example  table  below). 

❖  If  the  interviewee  makes  a  remark  about  a  performance  capability  use  the  remarks  sheet  located  at 
the  end  of  the  packet  to  document  the  remark,  use  the  section  letter  code  (Navigate  =  N,  Move  =  M, 
etc)  and  performance  capability  number  to  identify  the  remark,  i.e.  N1  =  Navigate  -  Used  map  to 
conduct  recon. 


DSTS 

Manned 
Module  Daily 
Checklist 

Observed 

Not  Observed 

Performed 

Did  Not  Perform 

Could  Not  Perform 

How  similar  was  the 
way  you  performed 
each  task  in  the  DSTS 
compared  to  the 
way  you  perform  it 
in  the  real  world? 

How  puickly  could 
you  perform  each 
task  in  the  DSTS 
compared  to  how 
quickly  you  can 
perform  it  in  the  real 
world? 

How  difficult  was  it 
for  you  to  perform 
each  task  in  the  DSTS 
compared  to  how 
difficult  it  is  in  the 

real  world? 

ID:_ATm_ 
DATE:  _19 

Mar  12_ 
Scenario: 

Day/ 

Offense/Wo 

odiand 

Start:  _0920_ 
Stop:  _1143_ 

Exactly  Like 

Very  Similar 

Somewhat  Similar 

Very  Different 

Completely  Different 

Much  Quicker 

Somewhat  Quicker 

About  the  Same 

Somewhat  Slower 

Much  Slower 

Much  Less  Difficult 

Somewhat  Less  Difficult 

About  the  Same 

Somewhat  More  Difficult 

Much  More  Difficult 

NAVIGATE 

1 

Used  map 
to  conduct 

recon 

X 

X 

1 

2 

Identified 

terrain 

features 

X 

X 

B 

2 

1 

D-2 


DSTS  Manned  Module 
Daily  Checklist 

Observed 

Not  Observed 

Performed 

Did  Not  Perform 

Could  Not  Perform 

How  similar  was  the  wav 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 

How  auickiv  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 

How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 

ID: 

Date: 

Scenario: 

Exactly  Like 

Very  Similar 

Somewhat  Similar 

Very  Different 

Completely  Different 

Much  Quicker 

Somewhat  Quicker 

About  the  Same 

Somewhat  Slower 

Much  Slower 

Much  Less  Difficult 

Somewhat  Less  Difficult 

About  the  Same 

Somewhat  More  Difficult 

Much  More  Difficult 

Start: 

Stoo: 

NAVIGATE 

1 

Used  map  to  conduct 

recon 

2 

Identified  terrain 

features 

3 

Determined  covered 

and  concealed  route 

4 

Determined  dir/dis  of 
travel 

5 

Used  compass  to 
determine  dir 

6 

Used  GPS  to  confirm 

location 

D-3 


Movement  Techniques 

1  Traveling 

2  Traveling  Overwatch 

3  Bounding  Overwatch 
Movement  Formations 

4  Squad  Column  (w/TMs 
in  wedge) 

5  Squad  Line 

6  Squad  File 


D-4 


DSTS  Manned  Module 
Daily  Checklist 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


Obstacles 


E 

I  £ 


"O  01 

01  O.  o 

o  z  2 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


u 

£ 

3  ° 

£  irt 

Q 


7 

Identified  an  obstacle 

8 

Negotiated  the 
obstacle 

9 

Bypassed  the  obstacle 

10 

Breached  the  obstacle 

11 

Marked  the 
passage/lane 

Individual  Movements 


12  Move  Forwards, 
Backwards,  Left,  Right 

13  Look  Right,  Left,  Behind 

14  Walk,  Run 


DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


15  Stand,  Lean  Left  and 
Right 

16  Kneel 

17  Prone, 


18  Low  Crawl 

19  Roll  Left  and  Right 

20  3-5  second  rush 


21  Climb 

22  Step  over 

23  Move  up  and  down 
stairs 

24  Open/close  doors 

25  Move  through  breach 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 
re 

iiJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 


DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


>  Si 

S  o 

Ul  4-1 

Si  o 

O  Z 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


01 

S 

o 

lO  01 

re  5 


3  ° 

£ 

Q 


26  Maintain  position 
within  formation 

27  Maintain  orientation 
within  simulation 

28  Maintain  balance  while 
moving 

29  Avoid  collisions 


Limited  Visibility 


30  Don  NVGs 


Fighting  Position  Selection 


DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


31  Occupy  fighting 
positions  w/cover, 
concealment,  and  fields 
of  fire 


Casualty  Evacuation 


32  Carry  casualty 

33  Drag  casualty 


Vehicle  Movement 


34  Mount  Vehicle 


35  Dismount  Vehicle 


COMMUNICATE 


>  Si 

S  o 

Ul  4-1 

Si  o 

o  z 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 


DSTS  Manned  Module 

Daily  Checklist 

ID: 

Date: 

Scenario: 

Start: 

Stoo: 

2 

Identify  own  fire  team 

3 

members 

Know  location  of  team 

members 

4 

Communicate  with  own 

fire  team 

5 

Communicate  with 

other  fire  team 

6 

Report  to  higher  (to 
Sqd/PIt  leader) 

7 

Use  hand  and  arm 
signals 

8 

Signal  w/pyrotechnics 

How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


-c  01 

01  Q. 

£  o 

o  z 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


u 

£ 

3  ° 

£ 

Q 


DSTS  Manned  Module 
Daily  Checklist 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


9  Communicate  with 
local  civilians 

10  Disperse  a  crowd  of 
civilians 

11  Move  through  a  crowd 
of  civilians 


SHOOT 


1  Detect  stationary  and 
moving  targets 


-C  01 

01  Q. 

£  o 

o  z 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 
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DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


E 


Start: 

Stop: 


enemy  fire 


4 


5 


6 


Select  primary  or 
secondary  weapon  (e.g 
Rifle,  AT-4,  Pistol, 
Battering  Ram,  or 
Binos) 

Achieve  correct  sight 
picture  with  sights  (iron 
or  optic) 

Aim  weapon  (apply 
lead  or  hold-off) 


ire  weapon 


Could  Not  Perform 


How  similar  was  the  way  How  quickly  could  you  How  difficult  was  it  for 
you  performed  each  task  perform  each  task  in  the  you  to  perform  each  task 
in  the  DSTS  compared  to  DSTS  compared  to  how  in  the  DSTS  compared  to 
the  way  you  perform  it  quickly  you  can  perform  how  difficult  it  is  in  the 
in  the  real  world?  it  in  the  real  world?  real  world? 
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DSTS  Manned  Module 
Daily  Checklist 


13  Switch  firing  hands 

14  Fire  in  short  bursts 

15  Reload  weapon 
(primary  or  alternate 
ammunition) 


Could  Not  Perform 


How  similar  was  the  wav 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 

How  auickiv  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 

How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 

Exactly  Like 

Very  Similar 

Somewhat  Similar 

Very  Different 

Completely  Different 

Much  Quicker 

Somewhat  Quicker 

About  the  Same 

Somewhat  Slower 

Much  Slower 

Much  Less  Difficult 

Somewhat  Less  Difficult 

About  the  Same 

Somewhat  More  Difficult 

Much  More  Difficult 

■ 

1 

1 

■ 

■ 
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DSTS  Manned  Module 
Daily  Checklist 


19  Emplace  breaching 
charge 

20  Detonate  breaching 
charge 

21  Select  grenade 

22  Throw  grenade 

23  Employ  claymore  mine 


Could  Not  Perform 


How  similar  was  the  wav 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 

How  auickiv  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 

How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 

Exactly  Like 

Very  Similar 

Somewhat  Similar 

Very  Different 

Completely  Different 

Much  Quicker 

Somewhat  Quicker 

About  the  Same 

Somewhat  Slower 

Much  Slower 

Much  Less  Difficult 

Somewhat  Less  Difficult 

About  the  Same 

Somewhat  More  Difficult 

Much  More  Difficult 

■ 

1 

■ 
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DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


24  Detonate  claymore 
mine 

25  Recover  claymore  mine 


React  to  lED 


26  Identify  visual 

indicators  of  lED/VBIED 


27  Alert  leadership  (dis, 
dir,  description) 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 
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DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


Call:  report  to  higher 
Cordon  area 
Control  ingress  and 
egress  of  cordoned 
area 

Move  to  rally  point 
Treat  and  evacuate 
casualties 


>  Si 

S  o 

Ul  4-1 

Si  o 

O  Z 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 


Engagement  Area  Development 


Fighting  Position  Selection 
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DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


2  Identify/assign 
alternate  position 

3  Identify/assign 
supplementary  position 


4  Identify  dead  space 
Obstacles 

5  Designated  location  of 
obstacle 

6  Determined  type  of 
obstacle 

7  Emplaced  concertina 


>  Si 

S  o 

Ul  4-1 

Si  o 

O  Z 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 
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DSTS  Manned  Module 
Daily  Checklist 


ID: _ 

Date: _ 

Scenario: 


Start: _ 

Stop: _ 


9  Reduced/retrieved 
obstacle 


Clear  Fields  of  Fire 


10  Drop  or  remove  trees 

11  Drop  or  remove  fences 

12  Rubble  buildings 


Adjacent  Unit  Coordination 


>  Si 

S  o 

Ul  4-1 

Si  o 

O  Z 


How  similar  was  the  way 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 


How  quickly  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 


How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 


re 

^  — 


u  >• 

re 

UJ  > 


01 

s 

o 

lO  01 

re  5 


3  ° 

£ 

Q 


I 
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DSTS  Manned  Module 
Daily  Checklist 

Observed 

Not  Observed 

Performed 

Did  Not  Perform 

Could  Not  Perform 

How  similar  was  the  wav 
you  performed  each  task 
in  the  DSTS  compared  to 
the  way  you  perform  it 
in  the  real  world? 

How  auickiv  could  you 
perform  each  task  in  the 
DSTS  compared  to  how 
quickly  you  can  perform 
it  in  the  real  world? 

How  difficult  was  it  for 
you  to  perform  each  task 
in  the  DSTS  compared  to 
how  difficult  it  is  in  the 
real  world? 

ID: 

Date: 

Scenario: 

Start: 

Stoo: 

Exactly  Like 

Very  Similar 

Somewhat  Similar 

Very  Different 

Completely  Different 

Much  Quicker 

Somewhat  Quicker 

About  the  Same 

Somewhat  Slower 

Much  Slower 

Much  Less  Difficult 

Somewhat  Less  Difficult 

About  the  Same 

Somewhat  More  Difficult 

Much  More  Difficult 

16 

Plan  Fires 

■ 

17 

Designate  TRPs 

■ 

18 

Communicate  TRPs  to 
Sqd  members 

1 
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APPENDIX  E 

SQUAD  MEMBER  POST-TRAINING  QUESTIONNAIRE 


E-1 


Squad  Member  Post-Training  Questionnaire 


Exercise  Preparation 


Please  assess  the  sufficiency  of  the  Training  Support  Materials, 
Training  Support  Personnel,  and  Training  Time,  Indicate  your 
assessment  with  one  “X”  per  question. 

Not  Observed 

Not  Performed 

Very  Sufficient 

Generally  Sufficient 

Somewhat  Sufficient 

Generally 

Insufficient 

Very  Insufficient 

Training  Support  Materials 

1 .  Was  there  enough  information  to  prepare  you  to  use  the  Manned 
Module? 

2 

6 

7 

1 

1 

1 

2.  Was  there  enough  information  to  prepare  you  to  use  the  Multi- 
funetion  Work  Station  (MFWS)  for  all  tasks? 

1 

1 

6 

6 

3 

1 

0 

3.  Were  the  training  support  materials  suffieient  to  guide  you  in 
mission  planning,  preparation,  rehearsals,  and  exeeution? 

1 

7 

7 

3 

1 

0 

DSTS  Training  Support  Personnel 

4.  Were  there  enough  trained  personnel  to  assist  you? 

13 

2 

1 

0 

2 

5.  Were  they  suffieiently  prepared  to  eonduet  the  exereise  preparation 
training? 

12 

3 

2 

1 

2 

6.  Were  they  suffieiently  knowledgeable  about  the  system  eapabilities 
to  eonduet  the  exereise  preparation  training? 

11 

5 

2 

1 

1 

7.  Were  they  suffieiently  knowledgeable  about  the  system  eapabilities 
to  eonduet  the  exereises  and  all  exeursions? 

11 

5 

2 

1 

1 

Pre-Training  Time 

8.  Was  there  sufficient  time  allocated  for  you  to  practice  with  the 
Manned  Module? 

2 

6 

7 

3 

1 

1 

9.  Was  there  sufficient  time  allocated  for  you  to  practice  with  the 
MFWS? 

1 

1 

4 

8 

3 

1 

1 

10.  Was  the  pre-training  time  too  long,  too  short,  or  just  enough? 

(circle  one) 

Too  long 

1 

Too  Short 

4 

Just 

Enough 

15 

E-2 


1 1 .  What  changes  would  you  make  to  the  pre-training,  i.e.  more  time  spent  on  ealibration,  or  more  time 
learning  how  to  navigate,  etc? 


More  time  learning  individual  movement  and  body  actions  operating  menu  controls  and  button. 


12.  Did  the  training  support  materials,  training  support  personnel,  or  training 
time  have  a  negative  impaet  on  your  ability  to  eomplete  the  exereises? 

(circle  one) 


No 

18 


13.  If  you  answered  “Yes”  to  question  12,  what  was  the  problem? 


Calibration  kept  going  out. 


Exercise  Execution 


Please  assess  the  sufficiency  of  the  Scenarios.  Indicate  your  assessment  with 
one  “X”  per  question. 

Very  Sufficient 

Generally  Sufficient 

Somewhat  Sufficient 

Generally 

Insufficient 

Very  Insufficient 

Scenarios 

14.  Was  the  length  (time)  of  the  seenario  suffieient  enough  for  the  training 
exereise? 

6 

10 

4 

0 

0 

15.  Was  the  seenario  suffieiently  realistie  enough  for  you  to  feel  immersed  in  the 
exereise? 

2 

8 

7 

2 

0 

16.  Was  the  seenario  suffieiently  eomplex  enough  to  ehallenge  you? 

4 

11 

4 

1 

0 

17.  Were  the  terrain,  landscape,  and  buildings  sufficiently  realistie  enough  not  to 
eause  a  distraetion  to  training? 

6 

7 

4 

3 

0 

18.  Were  the  audio  eues  distinet  enough  to  identify  the  loeation  of  enemy  forees? 

3 

6 

2 

7 

2 

19.  Were  the  visual  cues  distinet  enough  to  identify  the  loeation  of  enemy  forees? 

1 

12 

4 

2 

1 

20.  Were  there  any  simulator  or  simulation  distractions  to  mission 

Yes 

No 

execution?  (circle  one) 

5 

15 
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If  you  answered  Yes  to  question  21,  what  were  the  distraetions? 

Simulator  (Manned  Module  or  Mutlifunctional  Work  Station 

Manned  Module  system  calibration,  inability  to  manipulate  equipment  use  from  Multi-functional  Work 
Station  (e.g.  weapons,  vehicle),  and  inability  to  correctly  identify  team/squad  membership  avatar. 

Simulation  (VBS2  game  engine) 

Unreal,  (ability  to  go  through  solid  walls),  getting  stuck  at  or  unable  to  open  doors,  and  difficulty  breach 
obstacles. 


21 .  Did  you  notice  the  simulator  or  the  simulation  forcing  you  to  use  bad 

habits  i.e.  pointing  your  weapon  at  a  friendly  avatar  to  identify  who  it  is. 

Yes 

No 

or  performing  a  task  with  steps  out  of  sequence  (negative  training)? 

5 

15 

(circle  one) 

If  you  answered  Yes  to  question  22,  what  were  the  bad  habits  or  negative  training? 

Simulator  (Manned  Module  or  Mutlifunctional  Work  Station 

Pointing  weapon  at  other  team/squad  members  and  shooting  weapons  from  the  hip  because  of  optic  failures. 
Simulation  (VBS2  game  engine) 

No  comments 


DSTS  Overall  Assessment 


22.  How  do  you  think  your  performance  in  a  live  (real  world)  environment  setting  would  be  affected  by 
training  in  the  DSTS? 


General  indifferent,  responses  indicate  neither  a  strong  positive  or  negative  effect  on  real-world 
performance. 


23.  What  did  you  like  most  about  the  DSTS? 


Appreciation  of  capabilities  to  be  immersed  into  similar  experienced  real-world  environment  (e.g.  Iraq 
and  Afghanistan)  and  the  opportunity  to  practice  near  realistic  scenarios  (e.g.  force-on-force). 


24.  What  did  you  like  least  about  the  DSTS? 


Displeasure  with  the  weight  of  the  Manned  Module  equipment,  continuous  calibration  requirement,  and 
the  use  of  menu  buttons. 


25.  If  you  had  the  opportunity  to  improve  one  thing  about  this  DSTS,  what  would  it  be? 


Suggestions  recommend  reducing  the  weight  of  the  Manned  Module  equipment  and  improving  the 
interface  between  the  system  and  its  toggles,  switches,  and  menu  buttons. 


Please  indicate  your  agreement/disagreement  with  the  following  questions 
with  one  “X”  per  question. 

Strongly  Agree 

Agree 

Neither  Agree  or 
Disagree 

Disagree 

Strongly  Disagree 

DSTS  Fielding 

26.  This  DSTS  is  ready  for  fielding  aeross  the  entire  U.S.  Army  now,  no  further 
improvements  are  required. 

0 

4 

7 

7 

2 

27.  Even  though  this  DSTS  has  faults  and  things  requiring  eorreetion,  it  is  ‘good 
enough’  to  field  aeross  the  entire  U.S.  Army. 

3 

11 

5 

1 

0 

28.  This  DSTS  has  too  many  things  wrong  with  it  to  consider  fielding  it  across 
the  entire  U.S.  Army  unless  and  until  those  things  are  corrected. 

2 

3 

5 

10 

0 

Thank  you  for  your  partieipation 
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APPENDIX  F 

LEADER  POST-TRAINING  QUESTIONNAIRE 
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Platoon  Leader  DSTS  Training  Questionnaire 


Please  assess  the  sufficiency  of  the  Training  Support 
Materials,  Training  Support  Personnel,  and  Training 
Time.  Indicate  your  assessment  with  one  “X”  per 
question. 

Not  Observed 

Not  Performed 

Very  Sufficient 

Generally  Sufficient 

Somewhat  Sufficient 

Generally 

Insufficient 

Very  Insufficient 

Training  Support  Materials 

1 .  Was  there  enough  information  to  prepare  you  to  use  the 
Manned  Module? 

X 

2.  Was  there  enough  information  to  prepare  you  to  use  the 
Multi-function  Work  Station  (MFWS)  for  all  tasks? 

X 

3.  Were  the  training  support  materials  sufficient  to  guide  you  in 
mission  planning,  preparation,  rehearsals,  and  execution? 

X 

DSTS  Training  Support  Personnel 

4.  Were  there  enough  trained  personnel  to  assist  you? 

X 

5.  Were  they  sufficiently  prepared  to  conduct  the  exercise 
preparation  training? 

X 

6.  Were  they  sufficiently  knowledgeable  about  the  system 
capabilities  to  conduct  the  exercise  preparation  training? 

X 

7.  Were  they  sufficiently  knowledgeable  about  the  system 
capabilities  to  conduct  the  exercises  and  all  excursions? 

X 

Pre-Training  Time 

8.  Was  there  sufficient  time  allocated  for  you  to  practice  with 
the  Manned  Module? 

X 

9.  Was  there  sufficient  time  allocated  for  you  to  practice  with 
the  MFWS? 

X 

10.  Did  the  training  support  materials,  training  support  personnel,  or 
pre-training  time  have  a  negative  impact  on  the  Squad’s  ability  to 
complete  the  exercises? 

Yes 

No 

Please  assess  the  sufficiency  of  the  Scenarios.  Indicate  your 
assessment  with  one  “X”  per  question. 

Very  Sufficient 

Generally  Sufficient 

Somewhat  Sufficient 

Generally 

Insufficient 

Very  Insufficient 

Scenarios 

F-2 


1 1 .  Was  the  length  (time)  of  the  scenario  sufficient  enough  for  the  training 
exercise? 

X 

12.  Was  the  scenario  realistic  enough  to  immerse  the  unit? 

X 

13.  Was  the  scenario  complex  enough  to  challenge  and  evaluate  a  unit? 

X 

14.  Were  the  terrain,  landscape,  and  buildings  sufficiently  realistic  enough 
not  to  cause  a  distraction  to  training? 

X 

15.  Was  the  opposing  force  ratio  sufficient  enough  to  evaluate  the  unit? 

X 

16.  Was  the  system  sufficiently  flexible  to  modify  the  scenario  to  exercise 
and  evaluate  additional  tasks? 

X 

17.  Were  there  any  simulator  or  simulation  distractions  to 
mission  execution,  (circle  one) 

Yes 

No 

18.  Were  there  any  simulator  or  simulation  distractions  to 
mission  execution,  (circle  one) 

Yes 

No 
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APPENDIX  G 

DSTS  MANNED  MODULE  CAPABILITIES  CHECKLIST  DIFFICULTY  RATINGS 
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DSTS  Manned  Module  Capabilities  Cheeklist  Diffieulty  Ratings 


Much 

About  the 

Much 

System  Capability 

Quicker 

Same 

Slower 

t 

Green 

Employ  elaymore  mines  and 

19 

1 

0 

34.3 

other  obstaeles 

Destroy  a  position  in  a  building 

with  Armored  Vehiele  direet 

23 

11 

8 

9 

fire 

Amber 

Platoon  oeeupies  Assault 

54 

128 

45 

54.83 

Position 

Conduet  Platoon  movement 

85 

243 

90 

115.78 

teehniques 

Move  over,  through,  around. 

24 

213 

79 

179.44 

obstaele 

Move  using  eover 

28 

234 

83 

197.86 

Conduet  taetieal  movement  to 

78 

325 

112 

208.8 

an  Assault  Position 

Move  under  direet  fire 

94 

445 

196 

266.13 

Reaet  to  indireet  fire  mounted 

60 

238 

108 

125.34 

and  dismounted 

Conduet  movement  during 
limited  visibility 

41 

284 

105 

221.36 

Enter  a  Building  in  Urban  Area 

99 

450 

167 

290.38 

Deteet,  Suppress,  Destroy, 
enemy  Anti- Armor  position 

71 

199 

70 

97.14 

Use  vehiele  to  seeure  eleared 

15 

18 

7.32 

portions  of  the  Objeetive 

D 

Move  to  a  Taotieal  Assembly 
Area  or  designated  area  short  of 

89 

371 

135 

230.82 

the  Defense  Position 

Assign  primary,  alternate,  and 

supplementary  Eighting 

25 

231 

81 

201.99 

Positions 

Identify  dead  spaee  in  seetor 

Seleet  eovered  and  eoneealed 

21 

208 

76 

181.7 

routes  between  primary, 
alternate,  and  supplementary 

24 

175 

79 

126.05 

Defense  Positions 
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Much 

About  the 

Much 

System  Capability 

Quicker 

Same 

Slower 

t 

Tied  in  with  unit  to  the  left  and 

33 

251 

85 

210.8 

right 

Place  obstacles  concealed  from 

64 

210 

106 

89.2 

enemy  observation 

Clear  fields  of  lire 

27 

45 

4 

33.34 

Conduct  a  squad  combat  patrol. 

111 

492 

203 

294.23 

Engage  targets  with  M249 

90 

400 

204 

212.55 

SAW 

Engage  targets  with  M4 
Carbine. 

90 

400 

204 

212.55 

Engage  multiple  machine  gun 

135 

600 

306 

318.83 

targets 

Red 

Requests  direet  and/or  indirect 

22 

56 

74 

27.53 

fires 

Undetermine 

React  to  lED  while  dismounted 

0 

10 

1 

16.55 

Select  temporary  Eighting 
Position 

0 

11 

6 

10.71 

Use  Armored  Vehicle  to 
provide  eover  dismounted 

14 

17 

10 

1.8 

movement. 

Send  Situation  Reports  to 

11 

f. 

4 

3.71 

higher  headquarters 

\j 

Perform  voiee  communications 

11 

6 

3 

4.9 

Use  visual  signaling  technique 

2 

1 

1 

0.5 

Evaeuate  casualties  to  a  safe 

18 

10 

10.9 

loeation 

3 

Employ  smoke 

3 

1 

0 

3.5 

Engage  targets  with  Tank  main 

9 

5 

15 

5.24 

gun 

Interact  with  civilian 

0 

0 

0 

0 

Disperse  a  erowd  of  civilians. 

0 

4 

0 

8 

Move  through  a  crowd  of 

0 

4 

0 

8 

civilians. 

Armored  Eighting  Vehicle 
carries  Soldiers  to  dismount 

0 

12 

0 

24 

location 

Use  Armored  Vehicle  to 

provide  cover  fires  to  Assault  a 

18 

19 

16 

0.26 

building  and  clear  a  building 
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System  Capability _ 

Use  Armored  Vehiele  to  breaeh 

walls 

Conduct  Breach  during  an 
Assault 

Conduct  a  Breach  during  Urban 
operation 

Use  vehicle  to  establish  a  Road 
Block  or  barricade  to  isolate 
Objective  or  building 


Much 

Quicker 

About  the 
Same 

Much 

Slower 

23 

20 

16 

0 

5 

3 

3 

9 

2 

10 

6 

2 

t 

1.25 

4.75 

6.14 

5.33 
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APPENDIX  H 

BOLD  QUEST  -  DSTS  PERFORMANCE  CAPABILITIES  CHECKLIST 


H-1 


Bold  Quest  DSTS  Performance 


Step  over 


0 


0 


6 


Checklist  with  Response  Data 


Accuracy  Ratings  Difficulty  Ratings 


s 

3 

s 

« 

3 

o 

CZ3 

V 

u 

5 

o 

O 

s 

IZJ 

V 

o 

.fl 

o 

s 

.fi 

s 

s 

5 

© 

o 

O 

o 

< 

h4 

.fi 

0  0  0  0  0  0 

1  2  6  6  3  0 

0  0  0  0  0  0 

0  0  0  0  0  0 

6  3  0  0  6  3 

0  0  9  3  0  6 

0  3  6  3  0  6 

0  3  6  0  0  9 

0  0  4  1  0  3 

0  0  6  2  0  4 
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Similarity  Ratings  Accuracy  Ratings  Difficulty  Ratings 


Performance  Capability 

Exactly  Like 

Somewhat  Similar 

Completely 

Different 

More  Accurate 

About  the  Same 

Less  Accurate 

Less  Difficult 

About  the  Same 

More  Difficult 

Move  up  and  down  stairs 

0 

3 

2 

0 

3 

2 

2 

3 

0 

Open/Close  doors 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Maintain  position  in  formation 

Maintain  orientation  within 

0 

5 

1 

0 

3 

1 

3 

2 

1 

3 

simulation 

0 

3 

6 

0 

8 

0 

3 

6 

Maintain  balance  while  moving 

3 

6 

0 

0 

9 

0 

0 

9 

0 

Avoid  collisions 

0 

0 

9 

0 

0 

9 

0 

0 

9 

Carry  casualties 

0 

0 

9 

0 

3 

6 

6 

3 

0 

Drag  casualties 

0 

0 

3 

0 

3 

0 

3 

0 

0 

Employ  virtual  radio  (Leaders 
only) 

0 

0 

3 

0 

0 

3 

0 

0 

3 

Identify  own  squad/fire  team 
members 

0 

6 

3 

0 

0 

9 

0 

6 

3 

Know  location  of  Squad/team 
members 

0 

3 

6 

3 

H-3 

0 

6 

0 

0 

9 

Similarity  Ratings 


0 


2 


7 


Somewhat  Similar 


Accuracy  Ratings 


Difficulty  Ratings 


0  3  6  0  0  9 
0  3  6  0  0  9 
0  0  0  0  0  0 
0  0  0  0  0  0 
0  3  6  0  0  9 
0  0  9  0  0  9 
0  0  9  0  0  9 
0  0  0  0  0  0 
0  0  0  0  0  0 
0  3  6  0  2  7 
0  2  7  0  2  7 
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Similarity  Ratings  Accuracy  Ratings  Difficulty  Ratings 


Performance  Capability 

Exactly  Like 

Somewhat  Similar 

Completely 

Different 

More  Accurate 

About  the  Same 

Less  Accurate 

Less  Difficult 

About  the  Same 

More  Difficult 

Fire  weapon 

0 

9 

0 

0 

9 

0 

0 

9 

0 

Reload  weapon 

0 

6 

3 

0 

6 

3 

3 

6 

0 

Correct  weapon  malfunction 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Engage  targets  from  the  prone 

0 

4 

5 

0 

3 

6 

0 

3 

6 

Engage  targets  from  kneeling 

0 

9 

0 

0 

3 

6 

0 

3 

6 

Engage  targets  from  standing 

3 

6 

0 

0 

3 

6 

0 

3 

6 

Select  pyrotechnic  (smoke  or 
flare) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Throw/shoot  pyrotechnic 
(smoke  or  flare) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Select  grenade 

0 

0 

9 

0 

0 

9 

0 

0 

9 

Throw  grenade 

0 

0 

9 

0 

0 

9 

0 

0 

9 

Adjust  indirect  fire 

0 

0 

0 

0 

H-5 

0 

0 

0 

0 

0 

Similarity  Ratings 


Qi 

U 

i«! 

Performance  Capability _ 

Identify  objects  0  6  0 


Interact  with  objects  0  0  9 


Somewhat  Similar 


More  Accurate 


Accuracy  Ratings 


Difficulty  Ratings 


0  3  3  0  3  3 

0  0  9  0  0  9 
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APPENDIX  I 

BOLD  QUEST  -  DSTS  AFTER-ACTION  REVIEW  CAPABILITIES  QUESTIONNAIRE 


I-l 


Team  Member  AAR  Questionnaire 

Indicate  how  effective  each  After  Action  Review  capability  was  in  providing  performance 
feedback  to  you.  Please  consider  the  entire  4-point  scale  before  making  your 
responses. 


1 .  Playback  capability  to  show  where  your  avatar  went  and  what  happened. 
12  3  4 

Not  Effective  Somewhat  Effective  Mostly  Effective  Very  Effective 


2.  Ability  to  pause  or  slow  the  playback. 

12  3  4 

Not  Effective  Somewhat  Effective  Mostly  Effective  Very  Effective 


3.  Ability  to  see  the  scenario  from  the  "God's-eye"  view. 

12  3  4 

Not  Effective  Somewhat  Effective  Mostly  Effective  Very  Effective 


4.  Ability  to  change  the  point-of-view  to  see  other  things  on  the  battlefield. 
12  3  4 

Not  Effective  Somewhat  Effective  Mostly  Effective  Very  Effective 
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APPENDIX  J 

BOLD  QUEST  -  DECISION  MAKING  QUESTIONNAIRE 


J-1 


Respond  to  the  questions  by  eireling  the  number  (1  through  4)  that  best  represents  your  opinion. 
Please  consider  the  entire  4-point  scale  before  making  your  responses. 


1 .  Does  using  this  Simulator*  provide  you  an  opportunity  to  practice  making  sound  tactical 
decisions? 


No  Opportunity 

Some  Opportunity 

Good  Opportunity 

Great  Opportunity 

DSTS 

3 

14 

1 

0 

Live 

8 

8 

2 

0 

2.  Does  training  with  this  Simulator  improve  your  ability  to  make  more  rapid  tactical 
decisions? 


Will  not  Improve 

May  not  Improve 

May  Improve 

Will  Improve 

DSTS 

5 

8 

5 

0 

Live 

7 

8 

3 

0 

3.  Does  training  with  this  Simulator  make  you  more  confident  in  your  ability  to  make  tactical 
decisions? 


Not  Confident 

Somewhat 

Confident 

More  Confident 

Very  Confident 

DSTS 

8 

6 

4 

0 

Live 

7 

10 

I 

0 

4.  How  challenging  is  the  overall  experience  provided  by  training  with  this  Simulator? 


Not  Challenging 

Somewhat 

Challenging 

Mostly 

Challenging 

Very  Challenging 

DSTS 

7 

6 

4 

1 

Live 

7 

9 

2 

0 

5.  Does  training  with  this  Simulator  have  a  valuable  impact  on  your  decision  making  skills? 


Not  Valuable 

Somewhat  Valuable 

Mostly  Valuable 

Very  Valuable 

DSTS 

7 

5 

4 

2 

Live 

8 

8 

2 

0 

6.  Does  training  with  this  Simulator  help  you  focus  on  critical  factors  that  influence  tactical 
decisions? 


Not  Helpful 

Somewhat  Helpful 

Mostly  Helpful 

Very  Helpful 

DSTS 

6 

6 

4 

2 

Live 

7 

8 

3 

0 
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7.  To  what  extent  does  this  Simulator  teaeh  you  something  new  about  deeision  making  that 
is  not  now  or  not  easily  eovered  in  normal  classroom  or  field  training? 


Nothing  New 

Somewhat  New 

Mostly  New 

Entirely  New 

DSTS 

8 

3 

5 

2 

Live 

6 

9 

3 

0 

8.  To  what  extent  will  training  with  this  Simulator  help  you  make  sound  tactical  decisions? 


Not  Helpful 

Somewhat  Helpful 

Mostly  Helpful 

Very  Helpful 

DSTS 

8 

3 

4 

3 

Live 

8 

6 

4 

0 

9.  To  what  extent  will  training  with  this  Simulator  allow  you  to  practice  the  types  of 
decisions  you  must  make  as  a  small  unit  leader? 


Not  Allow 

Somewhat  Allow 

Mostly  Allow 

Always  Allow 

DSTS 

3 

7 

5 

3 

Live 

7 

7 

4 

0 

10.  Would  training  with  this  Simulator  be  a  valuable  learning  experience? 


Not  Valuable 

Somewhat  Valuable 

Mostly  Valuable 

Very  Valuable 

DSTS 

5 

6 

2 

5 

Live 

7 

7 

4 

0 

*Note.  The  word  “Simulator”  was  replaced  with  the  word  “Training  Environment”  on  the 
version  given  to  the  live-training  group. 
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APPENDIX  K 

BOLD  QUEST  -  TRAINING  PREPARATION  QUESTIONNAIRE 


K-1 


Respond  to  the  questions  by  eireling  the  number  (1  through  4)  that  best  represents  your  opinion. 
Please  consider  the  entire  4-point  scale  before  making  your  responses. 


1 .  Did  the  Simulation  Training*  improve  your  performance  in  the  live  mission? 


1 

2 

3 

4 

NO 

IMPROVEMENT 

SOME 

IMPROVEMENT 

MODERATE 

IMPROVEMENT 

SIGNIFICANT 

IMPROVEMENT 

2.  Did  the  Simulation  Training  enable  you  to  execute  the  live  mission  more 
quickly? 

1 

2 

3 

4 

NOT  MORE 
QUICKLY 

SOMEWHAT  MORE 
QUICKLY 

MORE 

QUICKLY 

MUCH  MORE 
QUICKLY 

3.  Did  the  Simulation  Training  increase  your  awareness  of  your  role  in  the  live 

mission? 

I 

2 

3 

4 

NO 

SOME 

MODERATE 

SIGNIFICANT 

INCREASE 

INCREASE 

INCREASE 

INCREASE 

4.  Did  the  Simulation  Training  improve  the  mission  success/effectiveness  of  your 

Squad? 

1 

2 

3 

4 

NO 

SOME 

MODERATE 

SIGNIFICANT 

IMPROVEMENT 

IMPROVEMENT 

IMPROVEMENT 

IMPROVEMENT 

5.  Did  the  Simulation  Training  make  you 

more  prepared  for  the  live  mission? 

I 

2 

3 

4 

NOT  MORE 

SOMEWHAT  MORE 

MORE 

MUCH  MORE 

PREPARED 

PREPARED 

PREPARED 

PREPARED 

6.  Did  the  Simulation  Training  increase  your  awareness  of  what  to  expect  in  the 

live  mission? 

1 

2 

3 

4 

NO 

SOME 

MODERATE 

SIGNIFICANT 

INCREASE 

INCREASE 

INCREASE 

INCREASE 

*Note.  The  words  “Simulation  Training”  were  replaced  with  the  word  “training”  on  the  version 
given  to  the  live-training  group. 
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APPENDIX  L 

BOLD  QUEST  -  OPERATIONAL  REALISM  QUESTIONNAIRE  WITH  DATA 


L-1 


Please  assess  the  sufficiency  of  the  Scenarios.  Indicate  your  assessment  with  one  “X”  per  question. 

Very  Generally  Generally  Very 
Suflleient  Suflieient  Insuflieient  Insufiieient 


1.  Was  the  length  (time)  of  the  scenario  sufficient  enough  for  the 
training  exercise? 

22%  56%  22%  0%  DSTS* 

28%  61%  6%  6%  Live* 

2.  Was  the  scenario  sufficiently  realistic  enough  for  you  to  feel 
immersed  in  the  exercise? 

3.  Was  the  scenario  sufficiently  complex  enough  to  challenge  you? 

28%  22%  28%  22%  DSTS 

34%  56%  6%  6%  Live 

4.  Were  the  terrain,  landscape,  and  buildings  sufficiently  realistic 
enough  not  to  cause  a  distraction  to  training? 

5.  Were  the  audio  cues  distinct  enough  to  replicate  patterns  of  life? 

6.  Were  the  visual  cues  distinct  enough  to  replicate  patterns  of  life? 

7.  Were  the  visual  clues  distinct  enough  to  discriminate  non- 
combatants? 

22%  39%  22%  17%  DSTS 

28%  56%  6%  11%  Live 

8.  Were  the  visual  clues  distinct  enough  to  identify  key  individuals? 

9.  Were  the  audio  cues  distinct  enough  to  identify  the  location  of 
enemy  forces? 

10.  Were  the  visual  cues  distinct  enough  to  identify  the  location  of 
enemy  forces? 

11.  Was  the  opposing  force  ratio  sufficient  enough  to  evaluate  the  unit? 

22%  39%  33%  6%  DSTS 

17%  72%  0%  11%  Live 

Note :  DSTS  =  Dismounted  Soldier  Training  System  eondition;  Live  =  Live-training  eondition 
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